[libvirt] [RFC PATCH 00/16] Introduce vGPU mdev framework to libvirt

[Erik Skultety]

On Mon, Feb 06, 2017 at 04:44:37PM +0000, Daniel P. Berrange wrote:
> On Mon, Feb 06, 2017 at 01:19:42PM +0100, Erik Skultety wrote:
> > Finally. It's here. This is the initial suggestion on how libvirt might
> > interract with the mdev framework, currently only focussing on the non-managed
> > devices, i.e. those pre-created by the user, since that will be revisited once
> > we all settled on how the XML should look like, given we might not want to use
> > the sysfs path directly as an attribute in the domain XML. My proposal on the
> > XML is the following:
> > 
> > <hostdev mode='subsystem' type='mdev'>  
> >     <source>
> >         <!-- this is the host's physical device address -->
> >         <address domain='0x0000' bus='0x00' slot='0x00' function='0x00'>
> >         <uuid>vGPU_UUID<uuid>
> >     <source>
> >     <!-- target PCI address can be omitted to assign it automatically -->
> > </hostdev>
> > 
> > So the mediated device is identified by the physical parent device visible on
> > the host and a UUID which allows us to construct the sysfs path by ourselves,
> > which we then put on the QEMU's command line.
> > 
> > A few remarks if you actually happen to have a machine to test this on:
> > - right now the mediated devices are one-time use only, i.e. they have to be
> > recreated before every machine boot
> > - I wouldn't recommend assigning multiple vGPUs to a single domain
> > 
> > Once this series is sorted out, we can then continue with 'managed=yes' where
> > as Laine pointed out [1], we need to figure out how exactly should the
> > management layer hint libvirt which vGPU type should be used for device
> > instantiation.
> 
> You seem to be suggesting that managed=yes with mdev devices would
> cause create / delete of a mdev device from a specified parent.
> 
> This is rather different semantics from what managed=yes does with
> PCI device assignment today.  There the managed=yes flag is just
> about controlling host device driver attachment. ie whether libvirt
> will manually bind to vfio.ko, or expect the admin to have bound
> it to vfio.ko before hand. I think it is important to keep that
> concept as is for mdev too.

If the managed attribute was used with other devices beside PCI, then sure, we
should keep the concept, however, since only PCI devices support it and now we
have another device type that potentially might have a use for such an
attribute I think it's perfectly reasonable to alter the logic behind that
attribute in favor of the new possibilities to device management which mdev
framework is providing us with which in this case is dynamic creation and
removal of a mediated device.

> 
> While we're thinking of mdev purely in terms of KVM + vfio usage,
> it wouldn't suprise me if there ended up being non-KVM based
> use cases for mdev.
> 
> It isn't clear to me that auto-creation of mdev devices as a concept
> even belongs in the domain XML neccessarily.
> 
> Looking at two similar areas. For SRIOV NICs, in the domain XML
> you either specify an explicit VF to use, or you reference a
> libvirt virtual network. The latter takes care of dynamically
> providing VFs to VMs.  For NPIV, IIRC, the domain XML works
> similarly either taking an explicit vHBA, or referencing a
> storage pool to get one more dynamically.
> 
> Before we even consider auto-creation though, I think we need
> to have manual creation designed & integrated in the node device
> APIs.
> 

Yes, integrating mdev into the nodedev driver this way ^^ is definitely planned.

> So in terms of the domain XML, I think the only think we need
> to provide is the address of the pre-existing mdev device
> to be used. In this case "address" means the UUID. We should
> not need anything about the parent device AFAICT.
> 
> Regards,
> Daniel
> -- 
> |: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org              -o-             http://virt-manager.org :|
> |: http://entangle-photo.org       -o-    http://search.cpan.org/~danberr/ :|