[vfio-users] Passing though an USB card problems

[Ede Wolf]

Hello,

I am trying to pass through a PCIe USB card to a guest, instead of just 
the ports, due to very sensitive USB devices.
Despite the unbind being reported as successful, the booting of the 
guest fails with an error:

qemu-system-x86_64: -device vfio-pci,host=09:04.0,x-vga=off: vfio 
0000:09:04.0: Failed to set up TRIGGER eventfd signaling for interrupt 
INTX-0: VFIO_DEVICE_SET_IRQS failure: Device or resource busy

While I have not been able to find much about this error, I do have one 
additional device in the same iommu group, that I suspect to be the 
reason for the failure. However, somehow this device lacks a "driver" 
folder and therefor the "unbind" file, so I cannot unbind it.

I am not sure wether this additional device may be a bridge on the card 
itself or a sensitive part of the mainboard, however blindly executing an

echo 12d8 e111 > /sys/bus/pci/drivers/vfio-pci/new_id'

does not change the behaviour.

Any ideas on what I may be missing or how to possibly resolve this? Any 
information that I may have been missing to provide?

Thanks

Ede

Here the information about the other device in the same iommu group:

# lspci -n -s 0000:08:00.0 -v
08:00.0 0604: 12d8:e111 (rev 02) (prog-if 00 [Normal decode])
         Flags: bus master, fast devsel, latency 0, IRQ 16, NUMA node 0
         Bus: primary=08, secondary=09, subordinate=09, sec-latency=32
         I/O behind bridge: 00008000-00008fff [size=4K]
         Memory behind bridge: bf600000-bf6fffff [size=1M]
         Prefetchable memory behind bridge: None
         Capabilities: [80] PCI-X bridge device
         Capabilities: [90] Power Management version 3
         Capabilities: [a8] Subsystem: 0000:0000
         Capabilities: [b0] Express PCI-Express to PCI/PCI-X Bridge, MSI 00
         Capabilities: [f0] MSI: Enable- Count=1/1 Maskable- 64bit+
         Capabilities: [100] Advanced Error Reporting
         Capabilities: [150] Virtual Channel


The first three folders are the ids of the USB card in question:

# ls -l /sys/bus/pci/devices/0000\:08\:00.0/

drwxr-xr-x 4 root root    0 23. Nov 15:16 0000:09:04.0
drwxr-xr-x 4 root root    0 23. Nov 15:16 0000:09:04.1
drwxr-xr-x 4 root root    0 23. Nov 15:16 0000:09:04.2
-r--r--r-- 1 root root 4096 23. Nov 15:16 aer_dev_correctable
-r--r--r-- 1 root root 4096 23. Nov 15:16 aer_dev_fatal
-r--r--r-- 1 root root 4096 23. Nov 15:16 aer_dev_nonfatal
-r--r--r-- 1 root root 4096 23. Nov 15:16 ari_enabled
-rw-r--r-- 1 root root 4096 23. Nov 15:16 broken_parity_status
-r--r--r-- 1 root root 4096 23. Nov 15:16 class
-rw-r--r-- 1 root root 4096 23. Nov 15:16 config
-r--r--r-- 1 root root 4096 23. Nov 15:16 consistent_dma_mask_bits
-r--r--r-- 1 root root 4096 23. Nov 15:16 current_link_speed
-r--r--r-- 1 root root 4096 23. Nov 15:16 current_link_width
-rw-r--r-- 1 root root 4096 23. Nov 15:16 d3cold_allowed
-r--r--r-- 1 root root 4096 23. Nov 15:16 device
-r--r--r-- 1 root root 4096 23. Nov 15:16 devspec
-r--r--r-- 1 root root 4096 23. Nov 15:16 dma_mask_bits
-rw-r--r-- 1 root root 4096 23. Nov 15:16 driver_override
-rw-r--r-- 1 root root 4096 23. Nov 15:16 enable

And the succesfull unbind, also verifyable by lsub, that lacks 3 devices 
after unbind:


# systemctl status kvm_virtio_prepare.service
● kvm_virtio_prepare.service - Preparation for PCI Passthru
    Loaded: loaded (/etc/systemd/system/kvm_virtio_prepare.service; 
static; vendor preset: disabled)
   Process: 963 ExecStart=/usr/bin/sh -c echo "0000:09:04.0" > 
/sys/bus/pci/devices/0000:09:04.0/driver/unbind (code=exited, 
status=0/SUCCESS)
   Process: 964 ExecStart=/usr/bin/sh -c echo "0000:09:04.1" > 
/sys/bus/pci/devices/0000:09:04.1/driver/unbind (code=exited, 
status=0/SUCCESS)
   Process: 965 ExecStart=/usr/bin/sh -c echo "0000:09:04.2" > 
/sys/bus/pci/devices/0000:09:04.2/driver/unbind (code=exited, 
status=0/SUCCESS)
   Process: 966 ExecStart=/usr/bin/sh -c echo 12d8 e111 > 
/sys/bus/pci/drivers/vfio-pci/new_id (code=exited, status=0/SUCCESS)
   Process: 967 ExecStart=/usr/bin/sh -c echo 1106 3038 > 
/sys/bus/pci/drivers/vfio-pci/new_id (code=exited, status=0/SUCCESS)
   Process: 968 ExecStart=/usr/bin/sh -c echo 1106 3104 > 
/sys/bus/pci/drivers/vfio-pci/new_id (code=exited, status=0/SUCCESS)
   Process: 969 ExecStart=/usr/bin/sh -c chgrp kvm /dev/vfio/25 
(code=exited, status=0/SUCCESS)
   Process: 970 ExecStart=/usr/bin/sh -c chmod 0660 /dev/vfio/25 
(code=exited, status=0/SUCCESS)
  Main PID: 970 (code=exited, status=0/SUCCESS)