[vfio-users] How to set PCI address in "options vfio-pci" ?

[Alex Williamson]

On Thu, 17 May 2018 13:12:07 -0400
Kyle Marek <psppsn96 at gmail.com> wrote:

> On 05/17/2018 10:27 AM, Alex Williamson wrote:
> > On Thu, 17 May 2018 12:56:49 +0800
> > Eddie Yen <missile0407 at gmail.com> wrote:
> >  
> >> Hi everyone,
> >>
> >> IME, you can set VID:PID by adding "options vfio-pci" into conf file and
> >> put at modprobe.d
> >> So that VFIO can take over the device that had same VID:PID when boot.
> >>
> >> But how can I set this option by using PCI Address without using override
> >> script?  
> > There's no such module option, what's wrong with a script?  Part of the
> > reason we don't have such a module option is that PCI addresses are not
> > persistent, they can change based on devices added or removed, or even
> > motherboard BIOS updates.  Userspace has more flexibility for managing
> > this and is fully able to do so via initrd scripts and modprobe.d.
> > Thanks,
> >
> > Alex  
> 
> That's a good point.
> 
> However, I had an environment where I needed to pass through one of two
> devices with the same PCI IDs. I accomplished it with an install command
> in my modprobe config.
> 
> I can probably make this look at other factors before overriding the
> driver, but the following config might work well enough if the PCI IDs
> don't change:
> 
> # For each PCI device listed in the for-loop, override their assigned driver
> # to vfio-pci, for usage with PCI-passthrough.
> #
> # Please make sure to include vfio-pci in the initrd, or some other modules
> # will likely claim the devices first. This can be done on systems that use
> # initramfs-tools by adding "vfio-pci" to /etc/initramfs-tools/modules
> install vfio-pci for dev in 0000:03:00.0 0000:03:00.1; do echo vfio-pci > "/sys/bus/pci/devices/$dev/driver_override"; done; /sbin/modprobe --ignore-install vfio-pci $CMDLINE_OPTS
> 
> # Load vfio-pci before some other drivers so they don't claim devices first.
> # The pre-drm line should handle all graphics drivers.
> softdep snd_hda_intel pre: vfio-pci
> softdep drm pre: vfio-pci
> 
> 
> This worked well for me in dracut and initramfs-tools initrds with the
> vfio-pci module included.

Yes, this is a good demonstration of how userspace can easily
accomplish this through modprobe.d softdeps and custom install
options.  A more persistence focused version of the above might do the
same knowing that we're looking for all devices device downstream of
the root port at 0000:00:01.0, and would look something like:

install vfio-pci for dev in $(find /sys/devices/pci0000:00/0000:00:01.0/
-maxdepth 1 -mindepth 1 -type d -name "????:??:??.?"); do echo vfio-pci > $dev/driver_override; done; modprobe --ignore-install...

or even

for override in $(find /sys/devices/pci0000:00/0000:00:01.0/ -mindepth
2 -name driver_override); do echo vfio-pci > $override; done...

There are countless ways to do this, including ways that take advantage
of PCI slots information provided through ACPI if the system supports
it.  Thanks,

Alex