<div dir="ltr">I've gone through reinstalls and multiple guides (<a href="http://vfio.blogspot.com/2015/05/vfio-gpu-how-to-series-part-3-host.html">http://vfio.blogspot.com/2015/05/vfio-gpu-how-to-series-part-3-host.html</a> a few times) and I'm not sure what to do next, no matter what I try I always get the error code 43.<div><br></div><div>My setup. I have a "server" in my closet that has 2 nvidia cards in (a GeForce GT 430 to run the system, a GeForce GTX 760 dedicated to the VM). I've setup the GTX 760 to use the stub (see below). The processor I have is a 'Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz', so no integrated graphics on that and the motherboard doesn't have any integrated graphics either. The server runs Ubuntu Server 16.04.1, but has libvirt and qemu updated from a PPA (see versions below). The VM is a Win 10 64-bit Home install with the NVidia 375.70 drivers installed from their website.</div><div><br></div><div>I've verified that if I pull the GT 430 from the box that I can see the boot on the GTX 760. If I leave both cards in, I don't see anything on the GTX 760 ever. One thing to note is that I setup the VM using virt-manager 1.3.2 from a 16.10 machine, so in the XML it mentions "<span style="white-space:pre-wrap">machine='pc-i440fx-yakkety'". I'm not sure if that could be causing any issues, but I've been working on this for about a week and need a fresh set of eyes.</span></div><div><font face="monospace"><br></font></div><div><font face="monospace">---- lspci -nnk -----</font></div><div><font face="monospace">01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF108 [GeForce GT 430] [10de:0de1] (rev a1)<br></font></div><div><div><font face="monospace"><span class="Apple-tab-span" style="white-space:pre"> </span>Subsystem: ZOTAC International (MCO) Ltd. GF108 [GeForce GT 430] [19da:1167]</font></div><div><font face="monospace"><span class="Apple-tab-span" style="white-space:pre"> </span>Kernel driver in use: nouveau</font></div><div><font face="monospace"><span class="Apple-tab-span" style="white-space:pre"> </span>Kernel modules: nvidiafb, nouveau</font></div><div><font face="monospace">01:00.1 Audio device [0403]: NVIDIA Corporation GF108 High Definition Audio Controller [10de:0bea] (rev a1)</font></div><div><font face="monospace"><span class="Apple-tab-span" style="white-space:pre"> </span>Subsystem: ZOTAC International (MCO) Ltd. GF108 High Definition Audio Controller [19da:1167]</font></div><div><font face="monospace"><span class="Apple-tab-span" style="white-space:pre"> </span>Kernel driver in use: snd_hda_intel</font></div><div><font face="monospace"><span class="Apple-tab-span" style="white-space:pre"> </span>Kernel modules: snd_hda_intel</font></div><div><font face="monospace">02:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK104 [GeForce GTX 760] [10de:1187] (rev a1)</font></div><div><font face="monospace"><span class="Apple-tab-span" style="white-space:pre"> </span>Subsystem: Micro-Star International Co., Ltd. [MSI] GK104 [GeForce GTX 760] [1462:2847]</font></div><div><font face="monospace"><span class="Apple-tab-span" style="white-space:pre"> </span>Kernel driver in use: vfio-pci</font></div><div><font face="monospace"><span class="Apple-tab-span" style="white-space:pre"> </span>Kernel modules: nvidiafb, nouveau</font></div><div><font face="monospace">02:00.1 Audio device [0403]: NVIDIA Corporation GK104 HDMI Audio Controller [10de:0e0a] (rev a1)</font></div><div><font face="monospace"><span class="Apple-tab-span" style="white-space:pre"> </span>Subsystem: Micro-Star International Co., Ltd. [MSI] GK104 HDMI Audio Controller [1462:2847]</font></div><div><font face="monospace"><span class="Apple-tab-span" style="white-space:pre"> </span>Kernel driver in use: vfio-pci</font></div><div><font face="monospace"><span class="Apple-tab-span" style="white-space:pre"> </span>Kernel modules: snd_hda_intel</font></div></div><div><font face="monospace"><br></font></div><div><font face="monospace">----- package lists from server -----</font></div><div><div><font face="monospace">thomas@smitty:~$ dpkg -l | grep libvirt</font></div><div><font face="monospace">ii libvirt-bin 2.2.0-0~16.04~ppa0 amd64 programs for the libvirt library</font></div><div><font face="monospace">ii libvirt-clients 2.2.0-0~16.04~ppa0 amd64 Programs for the libvirt library</font></div><div><font face="monospace">ii libvirt-daemon 2.2.0-0~16.04~ppa0 amd64 Virtualization daemon</font></div><div><font face="monospace">ii libvirt-daemon-system 2.2.0-0~16.04~ppa0 amd64 Libvirt daemon configuration files</font></div><div><font face="monospace">ii libvirt0:amd64 2.2.0-0~16.04~ppa0 amd64 library for interfacing with different virtualization systems</font></div><div><font face="monospace">thomas@smitty:~$ dpkg -l | grep qemu</font></div><div><font face="monospace">ii ipxe-qemu 1.0.0+git-20150424.a25a16d-1ubuntu1 all PXE boot firmware - ROM images for qemu</font></div><div><font face="monospace">ii qemu-block-extra:amd64 1:2.6.2+dfsg-0~16.04~ppa0 amd64 extra block backend modules for qemu-system and qemu-utils</font></div><div><font face="monospace">ii qemu-kvm 1:2.6.2+dfsg-0~16.04~ppa0 amd64 QEMU Full virtualization</font></div><div><font face="monospace">ii qemu-system-common 1:2.6.2+dfsg-0~16.04~ppa0 amd64 QEMU full system emulation binaries (common files)</font></div><div><font face="monospace">ii qemu-system-x86 1:2.6.2+dfsg-0~16.04~ppa0 amd64 QEMU full system emulation binaries (x86)</font></div><div><font face="monospace">ii qemu-utils 1:2.6.2+dfsg-0~16.04~ppa0 amd64 QEMU utilities</font></div></div><div><font face="monospace"><br></font></div><div><font face="monospace">----- Current VM XML dump -----</font></div><div><a href="http://termbin.com/mwwd"><font face="monospace">http://termbin.com/mwwd</font></a></div><div><font face="monospace"><br></font></div><div><font face="monospace"><br></font></div><div><pre style="color:rgb(34,34,34);font-size:13.2px"><code>----- find /sys/kernel/iommu_groups/ -type l -----</code></pre><pre><code><font color="#222222"><span style="font-size:13.2px">/sys/kernel/iommu_groups/0/devices/0000:00:00.0 /sys/kernel/iommu_groups/1/devices/0000:00:01.0 /sys/kernel/iommu_groups/2/devices/0000:00:01.1 /sys/kernel/iommu_groups/3/devices/0000:00:02.0 /sys/kernel/iommu_groups/4/devices/0000:00:03.0 /sys/kernel/iommu_groups/5/devices/0000:00:03.2 /sys/kernel/iommu_groups/6/devices/0000:00:05.0 /sys/kernel/iommu_groups/6/devices/0000:00:05.2 /sys/kernel/iommu_groups/6/devices/0000:00:05.4 /sys/kernel/iommu_groups/7/devices/0000:00:11.0 /sys/kernel/iommu_groups/8/devices/0000:00:16.0 /sys/kernel/iommu_groups/9/devices/0000:00:19.0 /sys/kernel/iommu_groups/10/devices/0000:00:1a.0 /sys/kernel/iommu_groups/11/devices/0000:00:1b.0 /sys/kernel/iommu_groups/12/devices/0000:00:1c.0 /sys/kernel/iommu_groups/13/devices/0000:00:1c.1 /sys/kernel/iommu_groups/14/devices/0000:00:1c.2 /sys/kernel/iommu_groups/15/devices/0000:00:1c.3 /sys/kernel/iommu_groups/16/devices/0000:00:1c.4 /sys/kernel/iommu_groups/17/devices/0000:00:1c.5 /sys/kernel/iommu_groups/18/devices/0000:00:1c.7 /sys/kernel/iommu_groups/19/devices/0000:00:1d.0 /sys/kernel/iommu_groups/20/devices/0000:00:1e.0 /sys/kernel/iommu_groups/21/devices/0000:00:1f.0 /sys/kernel/iommu_groups/21/devices/0000:00:1f.2 /sys/kernel/iommu_groups/21/devices/0000:00:1f.3 /sys/kernel/iommu_groups/22/devices/0000:05:00.0 /sys/kernel/iommu_groups/23/devices/0000:01:00.0 /sys/kernel/iommu_groups/23/devices/0000:01:00.1 /sys/kernel/iommu_groups/24/devices/0000:02:00.0 /sys/kernel/iommu_groups/24/devices/0000:02:00.1 /sys/kernel/iommu_groups/25/devices/0000:07:00.0 /sys/kernel/iommu_groups/26/devices/0000:08:00.0 /sys/kernel/iommu_groups/27/devices/0000:09:00.0 /sys/kernel/iommu_groups/28/devices/0000:0a:00.0 /sys/kernel/iommu_groups/29/devices/0000:0b:00.0 /sys/kernel/iommu_groups/30/devices/0000:0c:00.0 /sys/kernel/iommu_groups/31/devices/0000:0d:00.0 /sys/kernel/iommu_groups/32/devices/0000:ff:08.0 /sys/kernel/iommu_groups/33/devices/0000:ff:09.0 /sys/kernel/iommu_groups/34/devices/0000:ff:0a.0 /sys/kernel/iommu_groups/34/devices/0000:ff:0a.1 /sys/kernel/iommu_groups/34/devices/0000:ff:0a.2 /sys/kernel/iommu_groups/34/devices/0000:ff:0a.3 /sys/kernel/iommu_groups/35/devices/0000:ff:0b.0 /sys/kernel/iommu_groups/35/devices/0000:ff:0b.3 /sys/kernel/iommu_groups/36/devices/0000:ff:0c.0 /sys/kernel/iommu_groups/36/devices/0000:ff:0c.1 /sys/kernel/iommu_groups/36/devices/0000:ff:0c.2 /sys/kernel/iommu_groups/36/devices/0000:ff:0c.3 /sys/kernel/iommu_groups/37/devices/0000:ff:0d.0 /sys/kernel/iommu_groups/37/devices/0000:ff:0d.1 /sys/kernel/iommu_groups/37/devices/0000:ff:0d.2 /sys/kernel/iommu_groups/37/devices/0000:ff:0d.3 /sys/kernel/iommu_groups/38/devices/0000:ff:0e.0 /sys/kernel/iommu_groups/38/devices/0000:ff:0e.1 /sys/kernel/iommu_groups/39/devices/0000:ff:0f.0 /sys/kernel/iommu_groups/39/devices/0000:ff:0f.1 /sys/kernel/iommu_groups/39/devices/0000:ff:0f.2 /sys/kernel/iommu_groups/39/devices/0000:ff:0f.3 /sys/kernel/iommu_groups/39/devices/0000:ff:0f.4 /sys/kernel/iommu_groups/39/devices/0000:ff:0f.5 /sys/kernel/iommu_groups/40/devices/0000:ff:10.0 /sys/kernel/iommu_groups/40/devices/0000:ff:10.1 /sys/kernel/iommu_groups/40/devices/0000:ff:10.2 /sys/kernel/iommu_groups/40/devices/0000:ff:10.3 /sys/kernel/iommu_groups/40/devices/0000:ff:10.4 /sys/kernel/iommu_groups/40/devices/0000:ff:10.5 /sys/kernel/iommu_groups/40/devices/0000:ff:10.6 /sys/kernel/iommu_groups/40/devices/0000:ff:10.7 /sys/kernel/iommu_groups/41/devices/0000:ff:13.0 /sys/kernel/iommu_groups/41/devices/0000:ff:13.1 /sys/kernel/iommu_groups/41/devices/0000:ff:13.4 /sys/kernel/iommu_groups/41/devices/0000:ff:13.5 /sys/kernel/iommu_groups/42/devices/0000:ff:16.0 /sys/kernel/iommu_groups/42/devices/0000:ff:16.1 /sys/kernel/iommu_groups/42/devices/0000:ff:16.2<br></span></font></code></pre></div></div>