[libvirt] Trying to debug "Received unexpected event 3" from libvirt

[Michal Privoznik]

On 30.12.2015 11:30, Yaniv Kaul wrote:
> Hi,
> 

Hey,

sorry for getting to you so late.

> I'm trying to debug this issue, which may be affecting my inability to
> perform live snapshot.
> 1. I'm not sure what 'Waking up a tragedian" in the debug log means - what
> exactly is a tragedian?


It's the thread that has issued the change state API (shutdown, reboot,
..) and it's waiting for confirmation on the monitor. For instance, mgmt
app issues virDomainPMSuspendForDuration() which in qemu driver is
implemented via some agent calls. So the flow is like this:

1) the control gets to qemuDomainPMSuspendForDuration()
2) libvirt does some checks and issues 'guest-suspend-disk' command (or
corresponding command to the selected target).
3) qemu-ga running inside guest tries (!) to suspend the guest (it may
not necessarily succeed)
4) meanwhile, as guest is writing something onto disk (saving its RAM -
but one is unable to tell from outside), the libvirt API is blocked
5) finally, guest kernel calls 'HALT' to which qemu responds by sending
libvirt 'RESET' event
6) the libvirt event loop finds out that an event occurred on the domain
monitor and calls callback
7) the callback will wake up the sleeping API if the event the API is
waiting for matches the one obtained on the monitor

> 2. In any case, it'd be great if the WARN would mention mon->await_event -
> is it the event libvirt is actually waiting for?

Sure, that would be much more helpful - mind posting a patch?

> 
> (Both from qemu/qemu_agent.c)
> 3. I reckon event 3 is QEMU_AGENT_EVENT_RESET ? (from qemu/qemu_agent.h)
> 

Correct.

> 4. I'm also getting 'End of file while reading data: Input/output error'
> messages, not sure what they mean yet.


Usually they mean crashing daemon. If you are able to get a stacktrace
please do share it somewhere.

Michal