[vfio-users] VM Won't boot with 2 PCIE 1x cards installed.

Fri Sep 9 01:21:10 UTC 2016

I am running Arch Linux, ACS patch on a Skylake system.  I have some 
audio pop and crackle so wanted to try passing a sound card instead of 
routing the audio from my monitor to my speakers.  I purchased an Asus 
Xonar PCI DGX.  Chip is CMI8788.  I can boot the host with it, or my USB 
card.  But not both.  Error I get is as follows.

Error starting domain: internal error: qemu unexpectedly closed the 
monitor: 2016-09-09T01:17:41.729237Z qemu-system-x86_64: -device 
vfio-pci,host=0e:00.0,id=hostdev3,bus=pci.0,addr=0x6: vfio: Error: 
Failed to setup INTx fd: Device or resource busy
2016-09-09T01:17:41.729854Z qemu-system-x86_64: -device 
vfio-pci,host=0e:00.0,id=hostdev3,bus=pci.0,addr=0x6: Device 
initialization failed

Traceback (most recent call last):
   File "/usr/share/virt-manager/virtManager/asyncjob.py", line 88, in 
cb_wrapper
     callback(asyncjob, *args, **kwargs)
   File "/usr/share/virt-manager/virtManager/asyncjob.py", line 124, in 
tmpcb
     callback(*args, **kwargs)
   File "/usr/share/virt-manager/virtManager/libvirtobject.py", line 83, 
in newfn
     ret = fn(self, *args, **kwargs)
   File "/usr/share/virt-manager/virtManager/domain.py", line 1404, in 
startup
     self._backend.create()
   File "/usr/lib/python2.7/site-packages/libvirt.py", line 1035, in create
     if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self)
libvirtError: internal error: qemu unexpectedly closed the monitor: 
2016-09-09T01:17:41.729237Z qemu-system-x86_64: -device 
vfio-pci,host=0e:00.0,id=hostdev3,bus=pci.0,addr=0x6: vfio: Error: 
Failed to setup INTx fd: Device or resource busy
2016-09-09T01:17:41.729854Z qemu-system-x86_64: -device 
vfio-pci,host=0e:00.0,id=hostdev3,bus=pci.0,addr=0x6: Device 
initialization failed

I added the address for all devices to my VFIO.conf under modprode.d 
using options vfio-pci ids= to hopefully leave them only for the guest.  
But I doubt that is helping because I can start them with one or the 
other just not both.  Other than the pop and crack occasionally the 
guest runs great.  Below are my IOMMU groupings with IDs.  I am running 
out of ideas, not that I am an expert and could use a bit of help.

IOMMU group 0
         00:00.0 Host bridge [0600]: Intel Corporation Skylake Host 
Bridge/DRAM Registers [8086:191f] (rev 07)
IOMMU group 1
         00:01.0 PCI bridge [0604]: Intel Corporation Skylake PCIe 
Controller (x16) [8086:1901] (rev 07)
IOMMU group 2
         00:01.1 PCI bridge [0604]: Intel Corporation Skylake PCIe 
Controller (x8) [8086:1905] (rev 07)
IOMMU group 3
         00:14.0 USB controller [0c03]: Intel Corporation Sunrise 
Point-H USB 3.0 xHCI Controller [8086:a12f] (rev 31)
         00:14.2 Signal processing controller [1180]: Intel Corporation 
Sunrise Point-H Thermal subsystem [8086:a131] (rev 31)
IOMMU group 4
         00:16.0 Communication controller [0780]: Intel Corporation 
Sunrise Point-H CSME HECI #1 [8086:a13a] (rev 31)
IOMMU group 5
         00:17.0 SATA controller [0106]: Intel Corporation Sunrise 
Point-H SATA controller [AHCI mode] [8086:a102] (rev 31)
IOMMU group 6
         00:1b.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H 
PCI Root Port #17 [8086:a167] (rev f1)
IOMMU group 7
         00:1b.3 PCI bridge [0604]: Intel Corporation Sunrise Point-H 
PCI Root Port #20 [8086:a16a] (rev f1)
IOMMU group 8
         00:1c.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H 
PCI Express Root Port #1 [8086:a110] (rev f1)
IOMMU group 9
         00:1c.4 PCI bridge [0604]: Intel Corporation Sunrise Point-H 
PCI Express Root Port #5 [8086:a114] (rev f1)
IOMMU group 10
         00:1c.5 PCI bridge [0604]: Intel Corporation Sunrise Point-H 
PCI Express Root Port #6 [8086:a115] (rev f1)
IOMMU group 11
         00:1c.7 PCI bridge [0604]: Intel Corporation Sunrise Point-H 
PCI Express Root Port #8 [8086:a117] (rev f1)
IOMMU group 12
         00:1d.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H 
PCI Express Root Port #9 [8086:a118] (rev f1)
IOMMU group 13
         00:1d.4 PCI bridge [0604]: Intel Corporation Sunrise Point-H 
PCI Express Root Port #13 [8086:a11c] (rev f1)
IOMMU group 14
         00:1f.0 ISA bridge [0601]: Intel Corporation Sunrise Point-H 
LPC Controller [8086:a145] (rev 31)
         00:1f.2 Memory controller [0580]: Intel Corporation Sunrise 
Point-H PMC [8086:a121] (rev 31)
         00:1f.3 Audio device [0403]: Intel Corporation Sunrise Point-H 
HD Audio [8086:a170] (rev 31)
         00:1f.4 SMBus [0c05]: Intel Corporation Sunrise Point-H SMBus 
[8086:a123] (rev 31)
IOMMU group 15
         00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet 
Connection (2) I219-V [8086:15b8] (rev 31)
IOMMU group 16
         01:00.0 VGA compatible controller [0300]: NVIDIA Corporation 
GF110 [GeForce GTX 560 Ti 448 Cores] [10de:1087] (rev a1)
         01:00.1 Audio device [0403]: NVIDIA Corporation GF110 High 
Definition Audio Controller [10de:0e09] (rev a1)
IOMMU group 17
         02:00.0 VGA compatible controller [0300]: NVIDIA Corporation 
GM204 [GeForce GTX 970] [10de:13c2] (rev a1)
         02:00.1 Audio device [0403]: NVIDIA Corporation GM204 High 
Definition Audio Controller [10de:0fbb] (rev a1)
IOMMU group 18
         04:00.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1083/1085 
PCIe to PCI Bridge [1b21:1080] (rev 04)
         05:04.0 Multimedia audio controller [0401]: C-Media Electronics 
Inc CMI8788 [Oxygen HD Audio] [13f6:8788]
IOMMU group 19
         06:00.0 PCI bridge [0604]: Intel Corporation DSL6540 
Thunderbolt 3 Bridge [Alpine Ridge 4C 2015] [8086:1578]
IOMMU group 20
         07:00.0 PCI bridge [0604]: Intel Corporation DSL6540 
Thunderbolt 3 Bridge [Alpine Ridge 4C 2015] [8086:1578]
IOMMU group 21
         07:01.0 PCI bridge [0604]: Intel Corporation DSL6540 
Thunderbolt 3 Bridge [Alpine Ridge 4C 2015] [8086:1578]
IOMMU group 22
         07:02.0 PCI bridge [0604]: Intel Corporation DSL6540 
Thunderbolt 3 Bridge [Alpine Ridge 4C 2015] [8086:1578]
IOMMU group 23
         07:04.0 PCI bridge [0604]: Intel Corporation DSL6540 
Thunderbolt 3 Bridge [Alpine Ridge 4C 2015] [8086:1578]
IOMMU group 24
         0a:00.0 USB controller [0c03]: Intel Corporation DSL6540 USB 
3.1 Controller [Alpine Ridge] [8086:15b6]
IOMMU group 25
         0c:00.0 Ethernet controller [0200]: Intel Corporation I211 
Gigabit Network Connection [8086:1539] (rev 03)
IOMMU group 26
         0d:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 
Serial ATA Controller [1b21:0612] (rev 02)
IOMMU group 27
         0e:00.0 USB controller [0c03]: Renesas Technology Corp. 
uPD720201 USB 3.0 Host Controller [1912:0014] (rev 03)