[libvirt-users] PCI passthrough and abstraction

[Laine Stump]

On 4/25/19 10:14 AM, Mauricio Tavares wrote:
> So I am reading through , and am wondering what is the difference between
> 
> <hostdev mode='subsystem' type='pci' managed='yes'>
> <hostdev>
> 
> and
> 
> <interface type='hostdev' managed='yes'>
> </interface>
> 
> if I am trying to give full access to a NIC? Which one exposes more of the card?

I also answered this on IRC, but just in case someone is looking through 
the email archives and comes across this message:

There are a couple of differences between <hostdev> and <interface 
type='hostdev'>:

1)  <hostdev> will work with "many" different devices (as long as they 
are a PCI endpoint device, and don't share an IOMMU group with other 
devices that must remain in use by the host), but <interface 
type='hostdev'> can only be used for a PCI device that is a Virtual 
Function (VF) of an SR-IOV capable network card (if you're unsure 
whether or not your device qualifies, then it almost definitely *doesn't*

2) In preparation for using VFIO device assignment to assign the device 
to a guest, both types of device will be automatically unbound from 
their host driver and bound to the vfio-pci driver *if "managed='yes'" 
is set*. In addition, <interface type='hostdev'> will set the MAC 
address (and optionally the vlan tag) on the VF device before assigning 
it to the guest. This isn't done when you use <hostdev>, the result 
being that your guest will end up with a network device that has a 
random MAC address that is different each time it is started.

So if you are assigning VFs from an SR-IOV netcard, then you *really* 
want to use <interface type='hostdev'>. For all other cases of PCI 
device assignment (including netcards that aren't SR-IOV VFs), you 
*must* use <hostdev>.

As for "managed='yes'" - if you are never using the device directly on 
the host, it's highly recommended that you bind the device to the 
vfio-pci driver during the host bootup, and set "managed='no'" in the 
XML - probably 80% or more of the bugs we encounter with device 
assignment happen when we're unbinding the device from the host net (or 
whatever) driver, and binding it to vfio-pci or vice-versa - all sorts 
of timing issues that are essentially out of our control.


> On Wed, Apr 24, 2019 at 1:13 PM Mauricio Tavares <raubvogel at gmail.com> wrote:
>>
>> When you pass a device in the pci chain (after virsh
>> nodedev-dettach'ing it from host) to the guest, how much is passed
>> without being emulated/abstracted?
> 
> _______________________________________________
> libvirt-users mailing list
> libvirt-users at redhat.com
> https://www.redhat.com/mailman/listinfo/libvirt-users
>