[libvirt] [Qemu-devel] qapi DEVICE_DELETED event issued *before* instance_finalize?!

[Alex Williamson]

On Mon, 5 Sep 2016 11:36:55 +0200
Paolo Bonzini <pbonzini at redhat.com> wrote:

> On 05/09/2016 11:23, Markus Armbruster wrote:
> >> >
> >> > On the other hand, it is clearly documented that the DEVICE_DELETED
> >> > event is sent as soon as guest acknowledges completion of device
> >> > removal. So libvirt's buggy if we'd follow documentation strictly. But
> >> > then again, I don't see much information value in "guest has detached
> >> > device but qemu hasn't yet" event. Libvirt would ignore such event.  
> > Unless I'm missing something, libvirt needs an event that signals "Guest
> > and QEMU are done with this device".  Current DEVICE_DELETED isn't.
> > 
> > Can we imagine a use for current DEVICE_DELETED, i.e. "Guest is done,
> > but QEMU isn't"?
> > 
> > Would anything break if we changed semantics of DEVICE_DELETED to what
> > libvirt actually needs?
> > 
> > If the answers are "no" and "no", let's do it.  
> 
> There is a subtle aspect of this.  After the current DEVICE_DELETED, the
> device id is not used any more.  So technically you could have
> 
>    device_add bar,id=foo
>    device_del foo
> 
>    // something in QEMU prevents the device from going away?
>    // for example there is a storage issue that blocks completion
>    // of a read(), and bar is a storage device
> 
>    device_add bar,id=foo
>    device_del foo
> 
>    // which foo is being deleted?  The old one or the new one?
>    event DEVICE_DELETED
> 
> DEVICE_DELETED does have a meaning: management cannot talk to the device
> anymore in QMP once it is raised.

It seems like this is just pointing out another flaw in the semantics
of DEVICE_DELETED, a device can linger without a device id, so there's
no way to reference it via QMP.  QEMU can't signal anything more about
the device, nor can the VM admin perform any further operations on it.
It's like detecting planets around distant stars, libvirt can't actually
see the device, it can only monitor the affects the device has on the
VM.  This is broken and it seems like the fix is to push both the
release of the device id and the DEVICE_DELETED notification until
after the instance_finalize callback.  Doesn't that solve the nuance
you've identified here as well?

> Technically what libvirt wants to know for VFIO is not whether the
> device is gone; it's whether the device's _backend_ (the VFIO file
> descriptor) is gone.  The device backend could have been a separate QOM
> object, but it isn't.
> 
> So perhaps we need a new event that is specific to VFIO?

This immediately sounds like the wrong path.  A) Why is this vfio
specific?  B) Without a device id, how are we going to signal an
event?  It seems that nobody actually cares about this interim event in
QEMU and releasing the device id prior to the actual device itself is
just as problematic as the premature signal itself.  Thanks,

Alex