[vfio-users] USB card prevents qemu finding its harddrive

Ede Wolf listac at nebelschwaden.de
Sat Dec 28 14:08:18 UTC 2019 

Hello,

thanks very much for continuing to help. Here is the lspci -k with the 
problematic card passed through:

root at archiso ~ # lspci -k
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
         Subsystem: Red Hat, Inc Qemu virtual machine
         Kernel modules: intel_agp
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
         Subsystem: Red Hat, Inc Qemu virtual machine
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE 
[Natoma/Triton II]
         Subsystem: Red Hat, Inc Qemu virtual machine
         Kernel driver in use: ata_piix
         Kernel modules: ata_piix, pata_acpi, ata_generic
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
         Subsystem: Red Hat, Inc Qemu virtual machine
         Kernel driver in use: piix4_smbus
         Kernel modules: i2c_piix4
00:02.0 VGA compatible controller: Red Hat, Inc. QXL paravirtual graphic 
card (rev 04)
         Subsystem: Red Hat, Inc QEMU Virtual Machine
         Kernel driver in use: qxl
         Kernel modules: qxl
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
         Subsystem: Red Hat, Inc Device 0001
         Kernel driver in use: virtio-pci
         Kernel modules: virtio_pci
00:04.0 SCSI storage controller: Red Hat, Inc Virtio block device
         Subsystem: Red Hat, Inc Device 0002
         Kernel driver in use: virtio-pci
         Kernel modules: virtio_pci
00:05.0 Parallel controller: Oxford Semiconductor Ltd Device c110
         Subsystem: Oxford Semiconductor Ltd Device c110
         Kernel driver in use: parport_pc
         Kernel modules: parport_pc
00:06.0 USB controller: VIA Technologies, Inc. VL805 USB 3.0 Host 
Controller (rev 01)
         Subsystem: VIA Technologies, Inc. VL805 USB 3.0 Host Controller
         Kernel driver in use: xhci_hcd
         Kernel modules: xhci_pci


And here without:

root at archiso ~ # lspci -k
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
         Subsystem: Red Hat, Inc Qemu virtual machine
         Kernel modules: intel_agp
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
         Subsystem: Red Hat, Inc Qemu virtual machine
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE 
[Natoma/Triton II]
         Subsystem: Red Hat, Inc Qemu virtual machine
         Kernel driver in use: ata_piix
         Kernel modules: ata_piix, pata_acpi, ata_generic
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
         Subsystem: Red Hat, Inc Qemu virtual machine
         Kernel driver in use: piix4_smbus
         Kernel modules: i2c_piix4
00:02.0 VGA compatible controller: Red Hat, Inc. QXL paravirtual graphic 
card (rev 04)
         Subsystem: Red Hat, Inc QEMU Virtual Machine
         Kernel driver in use: qxl
         Kernel modules: qxl
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
         Subsystem: Red Hat, Inc Device 0001
         Kernel driver in use: virtio-pci
         Kernel modules: virtio_pci
00:04.0 SCSI storage controller: Red Hat, Inc Virtio block device
         Subsystem: Red Hat, Inc Device 0002
         Kernel driver in use: virtio-pci
         Kernel modules: virtio_pci
00:05.0 Parallel controller: Oxford Semiconductor Ltd Device c110
         Subsystem: Oxford Semiconductor Ltd Device c110
         Kernel driver in use: parport_pc
         Kernel modules: parport_pc

Very much to my surprise the redhat scsi controller is available in both 
configurations. But during boot all virtio devices (here its just 1) are 
is still missing.

The boot menu without the USB card in question:

Select boot device:

1. DVD/CD [ata0-0: QEMU DVD-ROM ATAPI-4 DVD/CD] (ARCH_201701)
2. Virtio disk PCI:00:04.0
3. iPXE (PCI 00:03.0)
4. Legacy option rom

And with:

1. DVD/CD [ata0-0: QEMU DVD-ROM ATAPI-4 DVD/CD] (ARCH_201701)
2. iPXE (PCI 00:03.0)
3. Legacy option rom

The virtio device is missing, despite being available after boot.

Following now the command line. The newlines have been added just for 
this mail. It's of course a one liner. The last line represents the 
problematic device and is just added or removed:


/usr/bin/qemu-system-x86_64 -enable-kvm -machine pc,accel=kvm
-name WinXPMidi -uuid 8c0d584b-8330-4a8c-bae0-c83f8029d52f
-cpu pentium2,kvm=off -smp sockets=2,cores=1,threads=1
-rtc base=localtime -m 2048 -no-hpet -serial none -parallel none
-nic vde,sock=/run/vde/vswitch2,model=virtio-net-pci,mac=6a:4e:17:ab:99:e1
-vga qxl -k de -spice ipv4,port=5901,addr=172.16.37.25,disable-ticketing
-D /var/log/qemu/winxp.log -nodefaults
-drive 
file=/VM/xp.raw,media=disk,format=raw,if=virtio,aio=native,cache=none,discard=ignore,index=0 

-drive 
file=/data/ISO/KVM-Install-Images/archlinux-2017.01.01-dual.iso,media=cdrom,if=ide,index=0 

-boot order=d,menu=on
-device vfio-pci,host=02:00.0,x-vga=off,bus=pci.0,addr=0x5
-device vfio-pci,host=08:00.0,x-vga=off,bus=pci.0,addr=0x6



And here are my iommu groups. I'll more than happily provide any other 
missing information:

# journalctl -k | grep -i iommu

Dez 28 14:20:11 vmpc kernel: Command line: BOOT_IMAGE=../vmlinuz-linux 
rootfstype=btrfs rootflags=subvol=/ 
root=UUID=bcc82208-1638-4b2d-b2ab-e1079ab4874e ipv6.disable=1 
intel_iommu=on audit=0 mitigations=off isolcpus=7 pcie_aspm=off rw 
initrd=../intel-ucode.img,../initramfs-linux.img
Dez 28 14:20:11 vmpc kernel: Kernel command line: 
BOOT_IMAGE=../vmlinuz-linux rootfstype=btrfs rootflags=subvol=/ 
root=UUID=bcc82208-1638-4b2d-b2ab-e1079ab4874e ipv6.disable=1 
intel_iommu=on audit=0 mitigations=off isolcpus=7 pcie_aspm=off rw 
initrd=../intel-ucode.img,../initramfs-linux.img

Dez 28 14:20:11 vmpc kernel: DMAR: IOMMU enabled
Dez 28 14:20:11 vmpc kernel: DMAR-IR: IOAPIC id 3 under DRHD base 
0xfbffe000 IOMMU 0
Dez 28 14:20:11 vmpc kernel: DMAR-IR: IOAPIC id 0 under DRHD base 
0xbfffc000 IOMMU 1
Dez 28 14:20:11 vmpc kernel: DMAR-IR: IOAPIC id 2 under DRHD base 
0xbfffc000 IOMMU 1
Dez 28 14:20:11 vmpc kernel: iommu: Default domain type: Translated
Dez 28 14:20:11 vmpc kernel: pci 0000:00:00.0: Adding to iommu group 0
Dez 28 14:20:11 vmpc kernel: pci 0000:00:01.0: Adding to iommu group 1
Dez 28 14:20:11 vmpc kernel: pci 0000:00:01.1: Adding to iommu group 2
Dez 28 14:20:11 vmpc kernel: pci 0000:00:02.0: Adding to iommu group 3
Dez 28 14:20:11 vmpc kernel: pci 0000:00:02.1: Adding to iommu group 4
Dez 28 14:20:11 vmpc kernel: pci 0000:00:02.2: Adding to iommu group 5
Dez 28 14:20:11 vmpc kernel: pci 0000:00:02.3: Adding to iommu group 6
Dez 28 14:20:11 vmpc kernel: pci 0000:00:03.0: Adding to iommu group 7
Dez 28 14:20:11 vmpc kernel: pci 0000:00:03.1: Adding to iommu group 8
Dez 28 14:20:11 vmpc kernel: pci 0000:00:03.2: Adding to iommu group 9
Dez 28 14:20:11 vmpc kernel: pci 0000:00:03.3: Adding to iommu group 10
Dez 28 14:20:11 vmpc kernel: pci 0000:00:04.0: Adding to iommu group 11
Dez 28 14:20:11 vmpc kernel: pci 0000:00:04.1: Adding to iommu group 11
Dez 28 14:20:11 vmpc kernel: pci 0000:00:04.2: Adding to iommu group 11
Dez 28 14:20:11 vmpc kernel: pci 0000:00:04.3: Adding to iommu group 11
Dez 28 14:20:11 vmpc kernel: pci 0000:00:04.4: Adding to iommu group 11
Dez 28 14:20:11 vmpc kernel: pci 0000:00:04.5: Adding to iommu group 11
Dez 28 14:20:11 vmpc kernel: pci 0000:00:04.6: Adding to iommu group 11
Dez 28 14:20:11 vmpc kernel: pci 0000:00:04.7: Adding to iommu group 11
Dez 28 14:20:11 vmpc kernel: pci 0000:00:05.0: Adding to iommu group 12
Dez 28 14:20:11 vmpc kernel: pci 0000:00:05.2: Adding to iommu group 12
Dez 28 14:20:11 vmpc kernel: pci 0000:00:05.4: Adding to iommu group 12
Dez 28 14:20:11 vmpc kernel: pci 0000:00:11.0: Adding to iommu group 13
Dez 28 14:20:11 vmpc kernel: pci 0000:00:16.0: Adding to iommu group 14
Dez 28 14:20:11 vmpc kernel: pci 0000:00:16.1: Adding to iommu group 14
Dez 28 14:20:11 vmpc kernel: pci 0000:00:1a.0: Adding to iommu group 15
Dez 28 14:20:11 vmpc kernel: pci 0000:00:1c.0: Adding to iommu group 16
Dez 28 14:20:11 vmpc kernel: pci 0000:00:1c.1: Adding to iommu group 17
Dez 28 14:20:11 vmpc kernel: pci 0000:00:1c.3: Adding to iommu group 18
Dez 28 14:20:11 vmpc kernel: pci 0000:00:1c.4: Adding to iommu group 19
Dez 28 14:20:11 vmpc kernel: pci 0000:00:1c.7: Adding to iommu group 20
Dez 28 14:20:11 vmpc kernel: pci 0000:00:1d.0: Adding to iommu group 21
Dez 28 14:20:11 vmpc kernel: pci 0000:00:1e.0: Adding to iommu group 22
Dez 28 14:20:11 vmpc kernel: pci 0000:00:1f.0: Adding to iommu group 23
Dez 28 14:20:11 vmpc kernel: pci 0000:00:1f.2: Adding to iommu group 23
Dez 28 14:20:11 vmpc kernel: pci 0000:00:1f.3: Adding to iommu group 23
Dez 28 14:20:11 vmpc kernel: pci 0000:02:00.0: Adding to iommu group 24
Dez 28 14:20:11 vmpc kernel: pci 0000:08:00.0: Adding to iommu group 25
Dez 28 14:20:11 vmpc kernel: pci 0000:0b:00.0: Adding to iommu group 26
Dez 28 14:20:11 vmpc kernel: pci 0000:0d:00.0: Adding to iommu group 27
Dez 28 14:20:11 vmpc kernel: pci 0000:0e:00.0: Adding to iommu group 28
Dez 28 14:20:11 vmpc kernel: pci 0000:0f:00.0: Adding to iommu group 28
Dez 28 14:20:11 vmpc kernel: pci 0000:10:00.0: Adding to iommu group 29
Dez 28 14:20:11 vmpc kernel: pci 0000:11:00.0: Adding to iommu group 30
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:08.0: Adding to iommu group 31
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:09.0: Adding to iommu group 32
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0a.0: Adding to iommu group 33
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0a.1: Adding to iommu group 33
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0a.2: Adding to iommu group 33
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0a.3: Adding to iommu group 33
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0b.0: Adding to iommu group 34
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0b.3: Adding to iommu group 34
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0c.0: Adding to iommu group 35
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0c.1: Adding to iommu group 35
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0c.2: Adding to iommu group 35
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0c.3: Adding to iommu group 35
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0d.0: Adding to iommu group 36
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0d.1: Adding to iommu group 36
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0d.2: Adding to iommu group 36
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0d.3: Adding to iommu group 36
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0e.0: Adding to iommu group 37
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0e.1: Adding to iommu group 37
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0f.0: Adding to iommu group 38
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0f.1: Adding to iommu group 38
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0f.2: Adding to iommu group 38
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0f.3: Adding to iommu group 38
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0f.4: Adding to iommu group 38
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:0f.5: Adding to iommu group 38
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:10.0: Adding to iommu group 39
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:10.1: Adding to iommu group 39
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:10.2: Adding to iommu group 39
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:10.3: Adding to iommu group 39
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:10.4: Adding to iommu group 39
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:10.5: Adding to iommu group 39
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:10.6: Adding to iommu group 39
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:10.7: Adding to iommu group 39
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:13.0: Adding to iommu group 40
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:13.1: Adding to iommu group 40
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:13.4: Adding to iommu group 40
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:13.5: Adding to iommu group 40
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:16.0: Adding to iommu group 41
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:16.1: Adding to iommu group 41
Dez 28 14:20:11 vmpc kernel: pci 0000:7f:16.2: Adding to iommu group 41
Dez 28 14:20:11 vmpc kernel: pci 0000:80:02.0: Adding to iommu group 42
Dez 28 14:20:11 vmpc kernel: pci 0000:80:03.0: Adding to iommu group 43
Dez 28 14:20:11 vmpc kernel: pci 0000:80:04.0: Adding to iommu group 44
Dez 28 14:20:11 vmpc kernel: pci 0000:80:04.1: Adding to iommu group 44
Dez 28 14:20:11 vmpc kernel: pci 0000:80:04.2: Adding to iommu group 44
Dez 28 14:20:11 vmpc kernel: pci 0000:80:04.3: Adding to iommu group 44
Dez 28 14:20:11 vmpc kernel: pci 0000:80:04.4: Adding to iommu group 44
Dez 28 14:20:11 vmpc kernel: pci 0000:80:04.5: Adding to iommu group 44
Dez 28 14:20:11 vmpc kernel: pci 0000:80:04.6: Adding to iommu group 44
Dez 28 14:20:11 vmpc kernel: pci 0000:80:04.7: Adding to iommu group 44
Dez 28 14:20:11 vmpc kernel: pci 0000:80:05.0: Adding to iommu group 45
Dez 28 14:20:11 vmpc kernel: pci 0000:80:05.2: Adding to iommu group 45
Dez 28 14:20:11 vmpc kernel: pci 0000:80:05.4: Adding to iommu group 45
Dez 28 14:20:11 vmpc kernel: pci 0000:81:00.0: Adding to iommu group 46
Dez 28 14:20:11 vmpc kernel: pci 0000:81:00.1: Adding to iommu group 46
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:08.0: Adding to iommu group 47
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:09.0: Adding to iommu group 48
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0a.0: Adding to iommu group 49
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0a.1: Adding to iommu group 49
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0a.2: Adding to iommu group 49
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0a.3: Adding to iommu group 49
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0b.0: Adding to iommu group 50
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0b.3: Adding to iommu group 50
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0c.0: Adding to iommu group 51
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0c.1: Adding to iommu group 51
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0c.2: Adding to iommu group 51
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0c.3: Adding to iommu group 51
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0d.0: Adding to iommu group 52
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0d.1: Adding to iommu group 52
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0d.2: Adding to iommu group 52
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0d.3: Adding to iommu group 52
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0e.0: Adding to iommu group 53
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0e.1: Adding to iommu group 53
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0f.0: Adding to iommu group 54
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0f.1: Adding to iommu group 54
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0f.2: Adding to iommu group 54
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0f.3: Adding to iommu group 54
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0f.4: Adding to iommu group 54
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:0f.5: Adding to iommu group 54
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:10.0: Adding to iommu group 55
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:10.1: Adding to iommu group 55
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:10.2: Adding to iommu group 55
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:10.3: Adding to iommu group 55
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:10.4: Adding to iommu group 55
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:10.5: Adding to iommu group 55
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:10.6: Adding to iommu group 55
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:10.7: Adding to iommu group 55
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:13.0: Adding to iommu group 56
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:13.1: Adding to iommu group 56
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:13.4: Adding to iommu group 56
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:13.5: Adding to iommu group 56
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:16.0: Adding to iommu group 57
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:16.1: Adding to iommu group 57
Dez 28 14:20:11 vmpc kernel: pci 0000:ff:16.2: Adding to iommu group 57
Dez 28 14:20:11 vmpc kernel: AMD-Vi: AMD IOMMUv2 driver by Joerg Roedel
Dez 28 14:20:11 vmpc kernel: AMD-Vi: AMD IOMMUv2 functionality not 
available on this system





Am 27.12.19 um 10:16 schrieb leesteken at pm.me:
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Wednesday, December 25, 2019 12:38 PM, Ede Wolf <listac at nebelschwaden.de> wrote:
> 
>> Hello,
>>
>> thanks for you answer. Unfortunately it does not seem to resolve this
>> issue.
>>
>> Because, if I use if=ide instead of if=virtio, I do not have this issue.
>>
>> This holds true for both a 4.19 lts kernel as well as a more recent 5.4.6.
>>
>> Further, when using if=virtio, no matter what address I use for the
>> passed through cards, the the harddrives are always missing.
>> And in case I do use an already used address, qemu startup failes with
>> an error, that this address is already used by [net|vga...]
>>
>> Also, I do have one device, that works. If I exchange that device with
>> the problematic one, using the same bus/address, the harddrives (virtio
>> disk pci:00:04.0/00:05.0) are still missing.
>>
>> Works (parallel card):
>> -device vfio-pci,host=02:00.0,x-vga=off,bus=pci.0,addr=0x6
>>
>> Does not work (changed from 0a to 08 due to changed slot):
>> -device vfio-pci,host=08:00.0,x-vga=off,bus=pci.0,addr=0x6
>>
>> Tried all adresses from 0x1 to 0x9. Either startup of qemu failed or no
>> bootable harddrives were found.
>>
>> So it does not matter wether I pass through both devices or just the
>> problematic one, the harddrives are missing any way.
>>
>> In addition, it makes no difference, wether I specify one or f.e. three
>> drives (files). As soon as I add the problematic card, all are gone,
>> when using the virtio interface.
>>
>> Of course I may have made a mistake, so I am happy about suggestions or
>> corrections.
>>
> 
> I'm out of ideas, sorry. Maybe more details information will provide a hint. What are your IOMMU groups?
> Can you boot the VM using some Linux LiveCD (using IDE) and show the output of lspci -k (in both cases)?
> Please also providing the whole kvm command line used to start the VM (in both cases).
> 
>> Ede
>>
>> Am 22.12.19 um 13:41 schrieb leesteken at pm.me:
>>
>>> Hi,
>>> Adding a device might shift the other (virtual) devices including the disk controller, or the disk controller might conflict with the default address that is used when you use -device to add it, which might remove (or not add) the disk controller.
>>> Try adding ,addr=0x?? to the -device parameter where you select an unused address for the ??.
>>> You might have to select a (virtual) bus as well as part of the -device parameter.
>>> hope this helps, Arjen
>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>>> On Sunday, December 22, 2019 1:14 PM, Ede Wolf listac at nebelschwaden.de wrote:
>>>
>>>> So I've bought this rather expensive USB 3.0 controller, making sure it
>>>> was compatible to everything on this planet and having a native pcie v2
>>>> interface and now I am having the problem, as soon as it gets passed
>>>> through, qemu cannot find it's boot disk any more.
>>>> "Boot failed. Could not read the boot disk". The boot menu only shows
>>>> legacy option rom or pxe.
>>>> I have no remote idea, where the relation between those two could be. It
>>>> is not, that the start up of the VM itself is failing, because of a
>>>> misconfiguration, it's just, that after powering on of the guest, it
>>>> cannot find any harddrives any more and tries to pxe boot instead.
>>>> As soon as I remove:
>>>> -device vfio-pci,host=0a:00.0,x-vga=off
>>>> from the configuration, the guest starts just normal - with another card
>>>> still passed through. So it is not a general passthrough problem. And
>>>> I've had passed through two cards before.
>>>> Needless to say, the guest image lives on a regular sata harddrive and
>>>> is not attached to that card in any way.
>>>> In addition, it is the only device in its vfio group, making it an even
>>>> more simple config:
>>>> journalctl -k | grep -i iommu | grep 25
>>>> ========================================
>>>> Dez 22 11:50:17 kernel: pci 0000:0a:00.0: Adding to iommu group 25
>>>> lspci -s "0a:00.0" -n -v
>>>> =========================
>>>> 0a:00.0 0c03: 1106:3483 (rev 01) (prog-if 30 [XHCI])
>>>> Subsystem: 1106:3483
>>>> Flags: fast devsel, IRQ 16, NUMA node 0
>>>> Memory at bf700000 (64-bit, non-prefetchable) [size=4K]
>>>> Capabilities: [80] Power Management version 3
>>>> Capabilities: [90] MSI: Enable- Count=1/4 Maskable- 64bit+
>>>> Capabilities: [c4] Express Endpoint, MSI 00
>>>> Capabilities: [100] Advanced Error Reporting
>>>> Kernel driver in use: vfio-pci
>>>> Kernel modules: xhci_pci
>>>> How is it possible that this card affects the boot drive of qemu? Where
>>>> is the connection between "-drive file=/home/VM/guest.img" and "-device
>>>> vfio-pci"? And how to eventually debug this?
>>>> Since this behaviour is related to vfio-pci, I am hoping, this is still
>>>> the correct list, even though this seems to affect qemu as whole. In a
>>>> quite obscure, magical way, one might add.
>>>> Thanks very much for any help
>>>> Ede
>>>> P.S.: Preparation, even though the config should not be the issue:
>>>> The Delock 5x USB Card (VIA)
>>>> =============================
>>>> ExecStart=-/usr/bin/sh -c 'echo "0000:0a:00.0" >
>>>> /sys/bus/pci/devices/0000:0a:00.0/driver/unbind'
>>>> ExecStart=/usr/bin/sh -c 'echo 1106 3483 >
>>>> /sys/bus/pci/drivers/vfio-pci/new_id'
>>>> ExecStart=/usr/bin/sh -c 'chgrp kvm /dev/vfio/25'
>>>> ExecStart=/usr/bin/sh -c 'chmod 0660 /dev/vfio/25'
>>>> vfio-users mailing list
>>>> vfio-users at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/vfio-users
>>
>> vfio-users mailing list
>> vfio-users at redhat.com
>> https://www.redhat.com/mailman/listinfo/vfio-users
> 
>

More information about the vfio-users mailing list