[libvirt] [RFC PATCH 00/28] Enable multifunction pci hotplug
Daniel P. Berrangé
berrange at redhat.com
Thu Mar 15 15:35:19 UTC 2018
On Thu, Mar 15, 2018 at 08:47:32PM +0530, Shivaprasad G Bhat wrote:
>
>
> On 03/15/2018 08:03 PM, Daniel P. Berrangé wrote:
> > On Thu, Mar 15, 2018 at 07:54:47PM +0530, Shivaprasad G Bhat wrote:
> > >
> > > On 03/15/2018 03:31 PM, Daniel P. Berrangé wrote:
> > > > On Wed, Mar 14, 2018 at 10:44:30PM +0530, Shivaprasad G Bhat wrote:
> > > > > Hi All,
> > > > >
> > > > > I have revisited/rewritten my previously posted patches. Here is
> > > > > the RFC. Since this patchset is a complete rewrite, I am starting
> > > > > with v1 here.
> > > > >
> > > > > The semantics is as discussed before
> > > > > https://www.redhat.com/archives/libvir-list/2016-April/msg01057.html
> > > > >
> > > > > As I went on to refactor the code to support multifunction virtio devices,
> > > > > I realised the abort/cleanup path would be a nightmare there, in case of
> > > > > failures. So, dropped that attempt. The current RFC limits to the real
> > > > > practical use cases of Multifunction PCI hostdevices. All new test code
> > > > > to support multifunction PCI hostdevices and test cases are added to
> > > > > prove the functionality.
> > > > I guess I'm not really understanding the use case here. With SRIOV
> > > > devices, you can already choose between assigning either the physical
> > > > function (which gives the guest access to all virtual functions), or
> > > > to assign an arbitrary set of individiual functions to various guests.
> > > > Why do we need to be able to list many <hostdev> at the same time
> > > > when hotplugging to assign multiple functions.
> > > >
> > > > Basically can you provide a full description of the problem you are
> > > > trying to solve and why existing functionality isn't sufficient.
> > > Hi Daniel,
> > >
> > > This is for cards which may not necessarily be networking cards. Or may be a
> > > mix of
> > > networking and storage.
> > >
> > > Suppose, user has below card
> > > 0005:01:00.0 Ethernet controller: Emulex Corporation OneConnect NIC (Lancer)
> > > (rev 10)
> > > 0005:01:00.1 Ethernet controller: Emulex Corporation OneConnect NIC (Lancer)
> > > (rev 10)
> > > 0005:01:00.2 Ethernet controller: Emulex Corporation OneConnect NIC (Lancer)
> > > (rev 10)
> > > 0005:01:00.3 Ethernet controller: Emulex Corporation OneConnect NIC (Lancer)
> > > (rev 10)
> > > 0005:01:00.4 Fibre Channel: Emulex Corporation OneConnect FCoE Initiator
> > > (Lancer) (rev 10)
> > > 0005:01:00.5 Fibre Channel: Emulex Corporation OneConnect FCoE Initiator
> > > (Lancer) (rev 10)
> > Ok, so this is a device with many functions, but which isn't SRIOV
> > based, and the goal is to assign the physical device to the guest,
> > such that guest has all functions available.
> >
> > > If user wants to hotplug this card to guest, He has to detach all the
> > > functions from host driver,
> > > then hotplug 0005:01:00.0, 0005:01:00.1, so on individually. But, today with
> > > each hotplug
> > > of the function, each <hostdev> goes to different guest slot. Whereas, PCI
> > > requires all of
> > > them to be on the same slot. This is not supported on libvirt today.
> > >
> > > The multifunction cards cant be hotplugged to guest today with the
> > > individual
> > > <hostdev>, as the operation is queued by qemu till the function zero of
> > > guest slot is
> > > hotplugged. On function zero hotplug, the qemu sends out the event to guest
> > > for device probing where all the previously hotplugged functions from the
> > > same slot are discovered. So, grouping the <hostdev>s within the <devices>
> > > would become necessary to make the whole thing a single operation.
> > So IIUC, from the patches, if the user wants to assign the physical
> > device to the guest, they would need to provide XML that looked like
> > this to the virDomainAttachDevice() method:
> >
> > <devices>
> > <hostdev mode='subsystem' type='pci' managed='yes'>
> > <driver name='vfio'/>
> > <source>
> > <address domain='0x0000' bus='0x05' slot='0x1' function='0x0'/>
> > </source>
> > </hostdev>
> > <hostdev mode='subsystem' type='pci' managed='yes'>
> > <driver name='vfio'/>
> > <source>
> > <address domain='0x0000' bus='0x05' slot='0x1' function='0x1'/>
> > </source>
> > </hostdev>
> > <hostdev mode='subsystem' type='pci' managed='yes'>
> > <driver name='vfio'/>
> > <source>
> > <address domain='0x0000' bus='0x05' slot='0x1' function='0x2'/>
> > </source>
> > </hostdev>
> > <hostdev mode='subsystem' type='pci' managed='yes'>
> > <driver name='vfio'/>
> > <source>
> > <address domain='0x0000' bus='0x05' slot='0x1' function='0x3'/>
> > </source>
> > </hostdev>
> > <hostdev mode='subsystem' type='pci' managed='yes'>
> > <driver name='vfio'/>
> > <source>
> > <address domain='0x0000' bus='0x05' slot='0x1' function='0x4'/>
> > </source>
> > </hostdev>
> > <hostdev mode='subsystem' type='pci' managed='yes'>
> > <driver name='vfio'/>
> > <source>
> > <address domain='0x0000' bus='0x05' slot='0x1' function='0x5'/>
> > </source>
> > </hostdev>
> > </devices>
> >
> >
> > Where as if the device were SRIOV based, they would only have to
> > provide
> >
> > <device>
> > <hostdev mode='subsystem' type='pci' managed='yes'>
> > <driver name='vfio'/>
> > <source>
> > <address domain='0x0000' bus='0x05' slot='0x1' function='0x0'/>
> > </source>
> > </hostdev>
> > </device>
> >
> > for the guest to get access to all functions.
> >
> > I find this difference in behaviour and approach really unpleasant.
> >
> > I think that they user should only need to provide the the address
> > of the physical device, in both cases. At most perhaps we need a
> > new attribute multifunction="on" on the source address to tell
> > libvirt that it should attach all the functions, not just the
> > first
> >
> > <device>
> > <hostdev mode='subsystem' type='pci' managed='yes'>
> > <driver name='vfio'/>
> > <source>
> > <address domain='0x0000' bus='0x05' slot='0x1' function='0x0' mutlifunction="on"/>
> > </source>
> > </hostdev>
> > </device>
>
> But with this approach the user can not prevent few functions from being
> assigned
> to guest if he wants to. It will be all or none. The PCI requires only
> function zero to be
> present and so, partial assignment is expected to work.
IIUC, once you've assigned the device with function 0 to a guest,
no other guest or the host, can safely used the other functions
of the device, right ? So I just assumed that if you're going to
give some functions to the guest you might as well give them all,
as nothing else can use them.
> So user should have that control?
Is there a use case for only giving some of the them ?
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
More information about the libvir-list
mailing list