[libvirt] libxl driver does not notice domain killed with xl destroy

Marek Marczykowski-Górecki marmarek at invisiblethingslab.com
Fri Dec 7 15:47:49 UTC 2018


If one kills a domain with `xl destroy`, libvirt will not notice it and
still report the domain as running. Also trying to destroy the domain
through libvirt will fail. The only way to recover from such a situation
is to restart libvirt daemon.

While (I think) the right answer is "don't use xl destroy", libvirt
could do better.

libxl actually report such cases via LIBXL_EVENT_TYPE_DOMAIN_DEATH, but
it is ignored by libvirt's libxl driver. The problem is that
LIBXL_EVENT_TYPE_DOMAIN_DEATH is also reported when it's triggered by
libvirt (for example libxlDomainShutdownHandleDestroy, or
For a proper solution I need a race-free way for checking if given
LIBXL_EVENT_TYPE_DOMAIN_DEATH was caused by libvirt (and is already
handled - including setting appropriate domain object state etc).

A bit of context:

    libxlDomainDestroyInternal(libxlDriverPrivatePtr driver,
                               virDomainObjPtr vm)
        libxlDriverConfigPtr cfg = libxlDriverConfigGet(driver);
        int ret = -1;

        /* Unlock virDomainObj during destroy, which can take considerable
         * time on large memory domains.
        ret = libxl_domain_destroy(cfg->ctx, vm->def->id, NULL);

        return ret;

Example usage:
    static int
    libxlDomainDestroyFlags(virDomainPtr dom,
                            unsigned int flags)
        if (libxlDomainObjBeginJob(driver, vm, LIBXL_JOB_MODIFY) < 0)
            goto cleanup;

        if (virDomainObjCheckActive(vm) < 0)
            goto endjob;

        if (libxlDomainDestroyInternal(driver, vm) < 0) {
                           _("Failed to destroy domain '%d'"), vm->def->id);
            goto endjob;

        virDomainObjSetState(vm, VIR_DOMAIN_SHUTOFF,

        event = virDomainEventLifecycleNewFromObj(vm, VIR_DOMAIN_EVENT_STOPPED,

It's tricky, because domain object lock is released for the
libxl_domain_destroy() call time and I think there is no guarantee that
the lock will be taken back by libxlDomainDestroyInternal() before
LIBXL_EVENT_TYPE_DOMAIN_DEATH is reported (and possibly handled). Which means
I can't use domain object state for checking if libvirt already know
about this domain death, because it is set to VIR_DOMAIN_SHUTOFF, after
libxlDomainDestroyInternal() call. And while setting domain
object state twice (once from LIBXL_EVENT_TYPE_DOMAIN_DEATH handler,
then from libxlDomainDestroyInternal() caller) wouldn't be that bad
(only ugly), but emitting domain lifecycle event twice would be a major

Any ideas? Some internal flag set by libxlDomainDestroyInternal() just
before libxl_domain_destroy() call? Where would be a place for such

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20181207/7dea1eab/attachment-0001.sig>

More information about the libvir-list mailing list