[libvirt] Libvirt domain event usage and consistency

[Michal Privoznik]

On 25.11.2016 17:54, Roman Mohr wrote:
> On Fri, Nov 25, 2016 at 4:34 PM, Michal Privoznik <mprivozn at redhat.com>
> wrote:
> 
>> On 25.11.2016 14:38, Roman Mohr wrote:
>>> Hi,
>>>
>>> I recently started to use the libvirt domain events. With them I increase
>>> the responsiveness of my VM state wachers.
>>> In general it works pretty well. I just listen to the events and do a
>>> periodic resync to cope with missed events.
>>>
>>> While watching the events I ran into a few interesting situations I
>> wanted
>>> to share. The points 1-3 describe some minor issues or irregularities.
>>> Point 4 is about the fact that domain and state updates are not versioned
>>> which makes it very hard to stay in sync with libvirt when using events.
>>>
>>> My libvirt version is 1.2.18.4.
>>
>> This might be the root cause. I'm unable to see some of the scenarios
>> you're seeing. Have you tried the latest release (or even git HEAD) to
>> check whether all the scenarios you are describing still stand?
>>
> 
> Definitely better with latest HEAD but still it does not look completely
> right.
> 
>>
>>>
>>> 1) Event order seems to be weird on startup:
>>>
>>> When listening for VM lifecycle events I get this order:
>>>
>>> {"event_type": "Started", "timestamp": "2016-11-25T11:59:53.209326Z",
>>> "reason": "Booted", "domain_name": "generic", "domain_id":
>>> "8ff7047b-fb46-44ff-a4c6-7c20c73ab86e"}
>>> {"event_type": "Defined", "timestamp": "2016-11-25T11:59:53.435530Z",
>>> "reason": "Added", "domain_name": "generic", "domain_id":
>>> "8ff7047b-fb46-44ff-a4c6-7c20c73ab86e"}
>>>
>>> It is strange that a VM already boots before it is defined. Is this the
>>> intended order?
>>
>> I don't see this order so probable this is fixed upstream.
>>
> 
> On latest master a normal creation emits these events:
> 
> event 'lifecycle' for domain testvm: Resumed Unpaused
> event 'lifecycle' for domain testvm: Started Booted
> 
> The Resumed event looks wrong. Further I get no more Defined/Undefined
> events. Maybe they were removed?

Yes, they were removed. The Resumed event comes from qemu actually,
because libvirt starts qemu in paused mode so that we can do some setup
(e.g. place vcpu threads into cgroups) and only after that we can resume
guest CPUs and in fact let guest start. Once this is done we
deliberately emit Started event.

> 
> 
>>
>>>
>>> 2) Defining a VM with VIR_DOMAIN_START_PAUSED gives me this event order
>>
>> I don't think you can define a domain with that flag. What's the actual
>> action?
>>
> 
> That is the flag for the api, when using virsh using `--paused` does that.

Ah, that's for virsh create/start not virsh define. Anyway, this is no
longer the case with upstream, is it?

> 
> 
>>
>>>
>>> {"event_type": "Defined", "timestamp": "2016-11-25T12:02:44.037817Z",
>>> "reason": "Added", "domain_name": "core_node", "domain_id":
>>> "b9906489-6d5b-40f8-a742-ca71b2b84277"}
>>> {"event_type": "Resumed", "timestamp": "2016-11-25T12:02:44.813104Z",
>>> "reason": "Unpaused", "domain_name": "core_node", "domain_id":
>>> "b9906489-6d5b-40f8-a742-ca71b2b84277"}
>>> {"event_type": "Started", "timestamp": "2016-11-25T12:02:44.813733Z",
>>> "reason": "Booted", "domain_name": "core_node", "domain_id":
>>> "b9906489-6d5b-40f8-a742-ca71b2b84277"}
>>
>>
>> Interesting, so here is "defined" event delivered before the "started"
>> event. Also - where is "suspended" event?
>>
> 
> 
> With latest master the situation looks better. Now I see
> 
> event 'lifecycle' for domain testvm: Started Booted
> event 'lifecycle' for domain testvm: Suspended Paused

Again, both of these are deliberately emitted by libvirt and in fact I
think they reflect what is happening.

Michal