[vfio-users] group not bound

[Alex Williamson]

On Tue, 5 Jan 2021 11:20:20 -0500
Roger Lawhorn <rll at twc.com> wrote:

> Hello,
> 
> I recently had to reinstall my OS, but kept all my personal files.
> One of the things that needs to be resetup partially is qemu.
> 
> I am getting the following error when running my qemu4.0 script to start 
> win10:
> ./seabios-rtx2080.sh 24
> pid 2558's current affinity list: 0-23
> pid 2558's new affinity list: 0-7
> qemu4.0-system-x86_64: -device 
> vfio-pci,host=0a:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on,romfile=/media/dad/QEMU-SSD/qemu-wd/rtx.rom: 
> vfio 0000:0a:00.0: group 16 is not viable
> Please ensure all devices within the iommu_group are bound to their vfio 
> bus driver.
> pid 2558's current affinity list: 0-7
> pid 2558's new affinity list: 0-23
> 
> I had this before but cant remember how i Resolved it.
> 
> Here is my iommu group 16:
> for a in /sys/kernel/iommu_groups/*; do find $a -type l; done
> 
> /sys/kernel/iommu_groups/16/devices/0000:0a:00.2
> /sys/kernel/iommu_groups/16/devices/0000:0a:00.0
> /sys/kernel/iommu_groups/16/devices/0000:0a:00.3
> /sys/kernel/iommu_groups/16/devices/0000:0a:00.1
> 
> 
> 
> group 16 is my evga rtx 2080 video card
> it has 4 devices. video, audio, usb, and serial bus.
> 
> here is my cmdline:
> cat /proc/cmdline
> BOOT_IMAGE=/vmlinuz-5.0.0-32-generic root=/dev/mapper/vg-root ro 
> vfio-pci.ids=10de:1e04,10de:10f7 mitigations=off
> 
> 
> i only hold the video and audio as the usb and serial wont hold.
>  >lspci -nnk  
> 0a:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device 
> [10de:1e04] (rev a1)
>      Subsystem: eVga.com. Corp. Device [3842:2281]
>      Kernel driver in use: vfio-pci
>      Kernel modules: nvidiafb, nouveau, nvidia_drm, nvidia
> 0a:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:10f7] (rev a1)
>      Subsystem: eVga.com. Corp. Device [3842:2281]
>      Kernel driver in use: vfio-pci
>      Kernel modules: snd_hda_intel
> 0a:00.2 USB controller [0c03]: NVIDIA Corporation Device [10de:1ad6] 
> (rev a1)
>      Subsystem: eVga.com. Corp. Device [3842:2281]
>      Kernel driver in use: xhci_hcd
> 0a:00.3 Serial bus controller [0c80]: NVIDIA Corporation Device 
> [10de:1ad7] (rev a1)
>      Subsystem: eVga.com. Corp. Device [3842:2281]
>      Kernel driver in use: nvidia-gpu
>      Kernel modules: i2c_nvidia_gpu

Ultimately all these devices need to be bound to vfio-pci or unused
ones can be bound to pci-stub, default host kernel drivers are not an
option.  Have you tried driverctl as a means to bind them to vfio-pci?
The pci-stub driver is usually built statically into the kernel
allowing it to claim devices before modules, ie.
pci-stub.ids=10de:1ad6,10de:1ad7.  The only way you could have
previously left them bound to default host drivers would have been
forcing group separation with the ACS override patch (not recommended).
Thanks,

Alex