[vfio-users] Passing though an USB card problems

Sun Nov 24 13:49:34 UTC 2019

Thanks very much for you fast reply. Indeed it is an old card, I've had 
to set pcie_aspm=off to avoid the console being flooded with error 
messages.
I am mot even sure, wether there have been USB 2.0 Chips native for PCI 
express at all.

More over, I've tried to change the slot location following your advice, 
and it seems to only be recognized by the system in one particular slot. 
In others, neither dmesg nor lspci do not even mention it. Seem to have 
been lucky to pick that one on the first go.

Not sure, why this is so, but in general, the card itself seems to be 
problematic.

So I'll replace it

Thanks again

Ede

Am 23.11.19 um 16:54 schrieb Alex Williamson:
> On Sat, 23 Nov 2019 15:34:21 +0100
> Ede Wolf <listac at nebelschwaden.de> wrote:
> 
>> Hello,
>>
>> I am trying to pass through a PCIe USB card to a guest, instead of just
>> the ports, due to very sensitive USB devices.
>> Despite the unbind being reported as successful, the booting of the
>> guest fails with an error:
>>
>> qemu-system-x86_64: -device vfio-pci,host=09:04.0,x-vga=off: vfio
>> 0000:09:04.0: Failed to set up TRIGGER eventfd signaling for interrupt
>> INTX-0: VFIO_DEVICE_SET_IRQS failure: Device or resource busy
>>
>> While I have not been able to find much about this error, I do have one
>> additional device in the same iommu group, that I suspect to be the
>> reason for the failure. However, somehow this device lacks a "driver"
>> folder and therefor the "unbind" file, so I cannot unbind it.
>>
>> I am not sure wether this additional device may be a bridge on the card
>> itself or a sensitive part of the mainboard, however blindly executing an
>>
>> echo 12d8 e111 > /sys/bus/pci/drivers/vfio-pci/new_id'
>>
>> does not change the behaviour.
>>
>> Any ideas on what I may be missing or how to possibly resolve this? Any
>> information that I may have been missing to provide?
> 
> The USB device you're trying to assign actually appears to be
> conventional PCI, not PCIe.  The other device in the group is a
> PCIe-to-PCI/X bridge.  If this is a PCIe plugin card, it must be an old
> one or one attempting to use old conventional PCI stock of the USB host
> controller chip.  In any case, there's probably a error in dmesg
> concurrent with the QEMU error that indicates that the device cannot be
> configured with an exclusive interrupt.  I suspect the problem is that
> the USB host controller failed the test for disabling INTx, which means
> that we can't use a shared interrupt and failed to get an exclusive
> interrupt.  It's not easy to configure the host to allow the device an
> exclusive interrupt, it might be possible by moving the device to a
> different slot or disabling drivers for other devices that share the
> same interrupt line.  It's usually easier to get a new device to assign
> that isn't broken in this way.  Thanks,
> 
> Alex
> 
> 
>> Here the information about the other device in the same iommu group:
>>
>> # lspci -n -s 0000:08:00.0 -v
>> 08:00.0 0604: 12d8:e111 (rev 02) (prog-if 00 [Normal decode])
>>           Flags: bus master, fast devsel, latency 0, IRQ 16, NUMA node 0
>>           Bus: primary=08, secondary=09, subordinate=09, sec-latency=32
>>           I/O behind bridge: 00008000-00008fff [size=4K]
>>           Memory behind bridge: bf600000-bf6fffff [size=1M]
>>           Prefetchable memory behind bridge: None
>>           Capabilities: [80] PCI-X bridge device
>>           Capabilities: [90] Power Management version 3
>>           Capabilities: [a8] Subsystem: 0000:0000
>>           Capabilities: [b0] Express PCI-Express to PCI/PCI-X Bridge, MSI 00
>>           Capabilities: [f0] MSI: Enable- Count=1/1 Maskable- 64bit+
>>           Capabilities: [100] Advanced Error Reporting
>>           Capabilities: [150] Virtual Channel
>>
>>
>> The first three folders are the ids of the USB card in question:
>>
>> # ls -l /sys/bus/pci/devices/0000\:08\:00.0/
>>
>> drwxr-xr-x 4 root root    0 23. Nov 15:16 0000:09:04.0
>> drwxr-xr-x 4 root root    0 23. Nov 15:16 0000:09:04.1
>> drwxr-xr-x 4 root root    0 23. Nov 15:16 0000:09:04.2
>> -r--r--r-- 1 root root 4096 23. Nov 15:16 aer_dev_correctable
>> -r--r--r-- 1 root root 4096 23. Nov 15:16 aer_dev_fatal
>> -r--r--r-- 1 root root 4096 23. Nov 15:16 aer_dev_nonfatal
>> -r--r--r-- 1 root root 4096 23. Nov 15:16 ari_enabled
>> -rw-r--r-- 1 root root 4096 23. Nov 15:16 broken_parity_status
>> -r--r--r-- 1 root root 4096 23. Nov 15:16 class
>> -rw-r--r-- 1 root root 4096 23. Nov 15:16 config
>> -r--r--r-- 1 root root 4096 23. Nov 15:16 consistent_dma_mask_bits
>> -r--r--r-- 1 root root 4096 23. Nov 15:16 current_link_speed
>> -r--r--r-- 1 root root 4096 23. Nov 15:16 current_link_width
>> -rw-r--r-- 1 root root 4096 23. Nov 15:16 d3cold_allowed
>> -r--r--r-- 1 root root 4096 23. Nov 15:16 device
>> -r--r--r-- 1 root root 4096 23. Nov 15:16 devspec
>> -r--r--r-- 1 root root 4096 23. Nov 15:16 dma_mask_bits
>> -rw-r--r-- 1 root root 4096 23. Nov 15:16 driver_override
>> -rw-r--r-- 1 root root 4096 23. Nov 15:16 enable
>>
>> And the succesfull unbind, also verifyable by lsub, that lacks 3 devices
>> after unbind:
>>
>>
>> # systemctl status kvm_virtio_prepare.service
>> ● kvm_virtio_prepare.service - Preparation for PCI Passthru
>>      Loaded: loaded (/etc/systemd/system/kvm_virtio_prepare.service;
>> static; vendor preset: disabled)
>>     Process: 963 ExecStart=/usr/bin/sh -c echo "0000:09:04.0" >
>> /sys/bus/pci/devices/0000:09:04.0/driver/unbind (code=exited,
>> status=0/SUCCESS)
>>     Process: 964 ExecStart=/usr/bin/sh -c echo "0000:09:04.1" >
>> /sys/bus/pci/devices/0000:09:04.1/driver/unbind (code=exited,
>> status=0/SUCCESS)
>>     Process: 965 ExecStart=/usr/bin/sh -c echo "0000:09:04.2" >
>> /sys/bus/pci/devices/0000:09:04.2/driver/unbind (code=exited,
>> status=0/SUCCESS)
>>     Process: 966 ExecStart=/usr/bin/sh -c echo 12d8 e111 >
>> /sys/bus/pci/drivers/vfio-pci/new_id (code=exited, status=0/SUCCESS)
>>     Process: 967 ExecStart=/usr/bin/sh -c echo 1106 3038 >
>> /sys/bus/pci/drivers/vfio-pci/new_id (code=exited, status=0/SUCCESS)
>>     Process: 968 ExecStart=/usr/bin/sh -c echo 1106 3104 >
>> /sys/bus/pci/drivers/vfio-pci/new_id (code=exited, status=0/SUCCESS)
>>     Process: 969 ExecStart=/usr/bin/sh -c chgrp kvm /dev/vfio/25
>> (code=exited, status=0/SUCCESS)
>>     Process: 970 ExecStart=/usr/bin/sh -c chmod 0660 /dev/vfio/25
>> (code=exited, status=0/SUCCESS)
>>    Main PID: 970 (code=exited, status=0/SUCCESS)
>>
>>
>> _______________________________________________
>> vfio-users mailing list
>> vfio-users at redhat.com
>> https://www.redhat.com/mailman/listinfo/vfio-users
> 