[vfio-users] Passing though an USB card problems

Sat Nov 23 15:54:29 UTC 2019

On Sat, 23 Nov 2019 15:34:21 +0100
Ede Wolf <listac at nebelschwaden.de> wrote:

> Hello,
> 
> I am trying to pass through a PCIe USB card to a guest, instead of just 
> the ports, due to very sensitive USB devices.
> Despite the unbind being reported as successful, the booting of the 
> guest fails with an error:
> 
> qemu-system-x86_64: -device vfio-pci,host=09:04.0,x-vga=off: vfio 
> 0000:09:04.0: Failed to set up TRIGGER eventfd signaling for interrupt 
> INTX-0: VFIO_DEVICE_SET_IRQS failure: Device or resource busy
> 
> While I have not been able to find much about this error, I do have one 
> additional device in the same iommu group, that I suspect to be the 
> reason for the failure. However, somehow this device lacks a "driver" 
> folder and therefor the "unbind" file, so I cannot unbind it.
> 
> I am not sure wether this additional device may be a bridge on the card 
> itself or a sensitive part of the mainboard, however blindly executing an
> 
> echo 12d8 e111 > /sys/bus/pci/drivers/vfio-pci/new_id'
> 
> does not change the behaviour.
> 
> Any ideas on what I may be missing or how to possibly resolve this? Any 
> information that I may have been missing to provide?

The USB device you're trying to assign actually appears to be
conventional PCI, not PCIe.  The other device in the group is a
PCIe-to-PCI/X bridge.  If this is a PCIe plugin card, it must be an old
one or one attempting to use old conventional PCI stock of the USB host
controller chip.  In any case, there's probably a error in dmesg
concurrent with the QEMU error that indicates that the device cannot be
configured with an exclusive interrupt.  I suspect the problem is that
the USB host controller failed the test for disabling INTx, which means
that we can't use a shared interrupt and failed to get an exclusive
interrupt.  It's not easy to configure the host to allow the device an
exclusive interrupt, it might be possible by moving the device to a
different slot or disabling drivers for other devices that share the
same interrupt line.  It's usually easier to get a new device to assign
that isn't broken in this way.  Thanks,

Alex

> Here the information about the other device in the same iommu group:
> 
> # lspci -n -s 0000:08:00.0 -v
> 08:00.0 0604: 12d8:e111 (rev 02) (prog-if 00 [Normal decode])
>          Flags: bus master, fast devsel, latency 0, IRQ 16, NUMA node 0
>          Bus: primary=08, secondary=09, subordinate=09, sec-latency=32
>          I/O behind bridge: 00008000-00008fff [size=4K]
>          Memory behind bridge: bf600000-bf6fffff [size=1M]
>          Prefetchable memory behind bridge: None
>          Capabilities: [80] PCI-X bridge device
>          Capabilities: [90] Power Management version 3
>          Capabilities: [a8] Subsystem: 0000:0000
>          Capabilities: [b0] Express PCI-Express to PCI/PCI-X Bridge, MSI 00
>          Capabilities: [f0] MSI: Enable- Count=1/1 Maskable- 64bit+
>          Capabilities: [100] Advanced Error Reporting
>          Capabilities: [150] Virtual Channel
> 
> 
> The first three folders are the ids of the USB card in question:
> 
> # ls -l /sys/bus/pci/devices/0000\:08\:00.0/
> 
> drwxr-xr-x 4 root root    0 23. Nov 15:16 0000:09:04.0
> drwxr-xr-x 4 root root    0 23. Nov 15:16 0000:09:04.1
> drwxr-xr-x 4 root root    0 23. Nov 15:16 0000:09:04.2
> -r--r--r-- 1 root root 4096 23. Nov 15:16 aer_dev_correctable
> -r--r--r-- 1 root root 4096 23. Nov 15:16 aer_dev_fatal
> -r--r--r-- 1 root root 4096 23. Nov 15:16 aer_dev_nonfatal
> -r--r--r-- 1 root root 4096 23. Nov 15:16 ari_enabled
> -rw-r--r-- 1 root root 4096 23. Nov 15:16 broken_parity_status
> -r--r--r-- 1 root root 4096 23. Nov 15:16 class
> -rw-r--r-- 1 root root 4096 23. Nov 15:16 config
> -r--r--r-- 1 root root 4096 23. Nov 15:16 consistent_dma_mask_bits
> -r--r--r-- 1 root root 4096 23. Nov 15:16 current_link_speed
> -r--r--r-- 1 root root 4096 23. Nov 15:16 current_link_width
> -rw-r--r-- 1 root root 4096 23. Nov 15:16 d3cold_allowed
> -r--r--r-- 1 root root 4096 23. Nov 15:16 device
> -r--r--r-- 1 root root 4096 23. Nov 15:16 devspec
> -r--r--r-- 1 root root 4096 23. Nov 15:16 dma_mask_bits
> -rw-r--r-- 1 root root 4096 23. Nov 15:16 driver_override
> -rw-r--r-- 1 root root 4096 23. Nov 15:16 enable
> 
> And the succesfull unbind, also verifyable by lsub, that lacks 3 devices 
> after unbind:
> 
> 
> # systemctl status kvm_virtio_prepare.service
> ● kvm_virtio_prepare.service - Preparation for PCI Passthru
>     Loaded: loaded (/etc/systemd/system/kvm_virtio_prepare.service; 
> static; vendor preset: disabled)
>    Process: 963 ExecStart=/usr/bin/sh -c echo "0000:09:04.0" > 
> /sys/bus/pci/devices/0000:09:04.0/driver/unbind (code=exited, 
> status=0/SUCCESS)
>    Process: 964 ExecStart=/usr/bin/sh -c echo "0000:09:04.1" > 
> /sys/bus/pci/devices/0000:09:04.1/driver/unbind (code=exited, 
> status=0/SUCCESS)
>    Process: 965 ExecStart=/usr/bin/sh -c echo "0000:09:04.2" > 
> /sys/bus/pci/devices/0000:09:04.2/driver/unbind (code=exited, 
> status=0/SUCCESS)
>    Process: 966 ExecStart=/usr/bin/sh -c echo 12d8 e111 > 
> /sys/bus/pci/drivers/vfio-pci/new_id (code=exited, status=0/SUCCESS)
>    Process: 967 ExecStart=/usr/bin/sh -c echo 1106 3038 > 
> /sys/bus/pci/drivers/vfio-pci/new_id (code=exited, status=0/SUCCESS)
>    Process: 968 ExecStart=/usr/bin/sh -c echo 1106 3104 > 
> /sys/bus/pci/drivers/vfio-pci/new_id (code=exited, status=0/SUCCESS)
>    Process: 969 ExecStart=/usr/bin/sh -c chgrp kvm /dev/vfio/25 
> (code=exited, status=0/SUCCESS)
>    Process: 970 ExecStart=/usr/bin/sh -c chmod 0660 /dev/vfio/25 
> (code=exited, status=0/SUCCESS)
>   Main PID: 970 (code=exited, status=0/SUCCESS)
> 
> 
> _______________________________________________
> vfio-users mailing list
> vfio-users at redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users