[libvirt] [RFC PATCH] hostdev: add support for "managed='detach'"

[Daniel P. Berrange]

On Tue, Mar 15, 2016 at 02:21:35PM -0400, Laine Stump wrote:
> On 03/15/2016 01:00 PM, Daniel P. Berrange wrote:
> >On Mon, Mar 14, 2016 at 03:41:48PM -0400, Laine Stump wrote:
> >>Suggested by Alex Williamson.
> >>
> >>If you plan to assign a GPU to a virtual machine, but that GPU happens
> >>to be the host system console, you likely want it to start out using
> >>the host driver (so that boot messages/etc will be displayed), then
> >>later have the host driver replaced with vfio-pci for assignment to
> >>the virtual machine.
> >>
> >>However, in at least some cases (e.g. Intel i915) once the device has
> >>been detached from the host driver and attached to vfio-pci, attempts
> >>to reattach to the host driver only lead to "grief" (ask Alex for
> >>details). This means that simply using "managed='yes'" in libvirt
> >>won't work.
> >>
> >>And if you set "managed='no'" in libvirt then either you have to
> >>manually run virsh nodedev-detach prior to the first start of the
> >>guest, or you have to have a management application intelligent enough
> >>to know that it should detach from the host driver, but never reattach
> >>to it.
> >>
> >>This patch makes it simple/automatic to deal with such a case - it
> >>adds a third "managed" mode for assigned PCI devices, called
> >>"detach". It will detach ("unbind" in driver parlance) the device from
> >>the host driver prior to assigning it to the guest, but when the guest
> >>is finished with the device, will leave it bound to vfio-pci. This
> >>allows re-using the device for another guest, without requiring
> >>initial out-of-band intervention to unbind the host driver.
> >You say that managed=yes causes pain upon re-attachment and that
> >apps should use managed=detach to avoid it, but how do management
> >apps know which devices are going to cause pain ? Libvirt isn't
> >providing any info on whether a particular device id needs to
> >use managed=yes vs managed=detach, and we don't want to be asking
> >the user to choose between modes in openstack/ovirt IMHO. I think
> >thats a fundamental problem with inventing a new value for managed
> >here.
> 
> My suspicion is that in many/most cases users don't actually need for the
> device to be re-bound to the host driver after the guest is finished with
> it, because they're only going to use the device to assign to a different
> guest anyway. But because managed='yes' is what's supplied and is the
> easiest way to get it setup for assignment to a guest, that's what they use.
> 
> As a matter of fact, all this extra churn of changing the driver back and
> forth for devices that are only actually used when they're bound to vfio-pci
> just wastes time, and makes it more likely that libvirt and its users will
> reveal and get caught up in the effects of some strange kernel driver
> loading/unloading bug (there was recently a bug reported like this;
> unfortunately the BZ record had customer info in it, so it's not publicly
> accessible :-( )
> 
> So beyond making this behavior available only when absolutely necessary, I
> think it is useful in other cases, at the user's discretion (and as I
> implied above, I think that if they understood the function and the
> tradeoffs, most people would choose to use managed='detach' rather than
> managed='yes')

IIUC, in managed=yes mode we explicitly track whether the device was
originally attached to a host device driver. ie we only re-attach
the device to the host when guest shuts down, if it was attached to
the host at guest startup.

We already have a virNodeDeviceDetach() API that can be used to
detach a device from the host driver explicitly.

So applications can in fact already achieve what you describe in
terms of managed=detach, by simply calling virNodeDeviceDetach()
prior to starting the guest with cold plugged PCI devices / hotplugging
the PCI device.

IOW, even if we think applications should be using managed=detach,
they can already do so via existing libvirt APIs.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|