[libvirt] Trying to debug "Received unexpected event 3" from libvirt

[Yaniv Kaul]

On Fri, Jan 8, 2016 at 7:00 PM, Michal Privoznik <mprivozn at redhat.com>
wrote:

> On 30.12.2015 11:30, Yaniv Kaul wrote:
> > Hi,
> >
>
> Hey,
>
> sorry for getting to you so late.
>
> > I'm trying to debug this issue, which may be affecting my inability to
> > perform live snapshot.
> > 1. I'm not sure what 'Waking up a tragedian" in the debug log means -
> what
> > exactly is a tragedian?
>
>
> It's the thread that has issued the change state API (shutdown, reboot,
> ..) and it's waiting for confirmation on the monitor. For instance, mgmt
> app issues virDomainPMSuspendForDuration() which in qemu driver is
> implemented via some agent calls. So the flow is like this:
>
> 1) the control gets to qemuDomainPMSuspendForDuration()
> 2) libvirt does some checks and issues 'guest-suspend-disk' command (or
> corresponding command to the selected target).
> 3) qemu-ga running inside guest tries (!) to suspend the guest (it may
> not necessarily succeed)
> 4) meanwhile, as guest is writing something onto disk (saving its RAM -
> but one is unable to tell from outside), the libvirt API is blocked
> 5) finally, guest kernel calls 'HALT' to which qemu responds by sending
> libvirt 'RESET' event
> 6) the libvirt event loop finds out that an event occurred on the domain
> monitor and calls callback
> 7) the callback will wake up the sleeping API if the event the API is
> waiting for matches the one obtained on the monitor
>
> > 2. In any case, it'd be great if the WARN would mention mon->await_event
> -
> > is it the event libvirt is actually waiting for?
>
> Sure, that would be much more helpful - mind posting a patch?
>

Attached and tested. I can't post it properly, as the git is a bit 'dirty'
with some po files that I can't clean:

On branch workbranch
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   po/ar.po
        modified:   po/as.po
        modified:   po/bg.po
        modified:   po/bn.po
        modified:   po/bn_IN.po
        modified:   po/bs.po
        modified:   po/ca.po
        modified:   po/cs.po
        modified:   po/cy.po
        modified:   po/da.po
        modified:   po/de.po
        modified:   po/el.po
        modified:   po/en_GB.po
        modified:   po/es.po
        modified:   po/et.po
        modified:   po/fi.po
        modified:   po/fr.po
        modified:   po/gl.po
        modified:   po/gu.po
        modified:   po/he.po
        modified:   po/hi.po

...

 Thanks,
Y.


> >
> > (Both from qemu/qemu_agent.c)
> > 3. I reckon event 3 is QEMU_AGENT_EVENT_RESET ? (from qemu/qemu_agent.h)
> >
>
> Correct.
>
> > 4. I'm also getting 'End of file while reading data: Input/output error'
> > messages, not sure what they mean yet.
>
>
> Usually they mean crashing daemon. If you are able to get a stacktrace
> please do share it somewhere.
>
> Michal
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20160114/a3ba95bc/attachment-0001.htm>
-------------- next part --------------
diff --git a/src/qemu/qemu_agent.c b/src/qemu/qemu_agent.c
index f979f82..924c177 100644
--- a/src/qemu/qemu_agent.c
+++ b/src/qemu/qemu_agent.c
@@ -1241,7 +1241,7 @@ void qemuAgentNotifyEvent(qemuAgentPtr mon,
         }
     } else {
         /* shouldn't happen but one never knows */
-        VIR_WARN("Received unexpected event %d", event);
+        VIR_WARN("Received unexpected event %d (expected %d)", event, mon->await_event);
     }
 }