<div dir="ltr">Hi,<div> I'm having an insanely hard time trying to pass an SR-IOV Virtual Function (VF) to a QEMU VM.</div><div><br></div><div>I've followed the instructions at <a href="http://www.linux-kvm.org/page/10G_NIC_performance:_VFIO_vs_virtio">http://www.linux-kvm.org/page/10G_NIC_performance:_VFIO_vs_virtio</a></div><div>and I successfully managed to create one (or more) VFs on my ixgbe NIC.</div><div><br></div><div>My host kernel is Linux 4.10.8, patched with the ACS override patch, as suggested here</div><div><a href="https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF#Bypassing_the_IOMMU_groups_.28ACS_override_patch.29">https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF#Bypassing_the_IOMMU_groups_.28ACS_override_patch.29</a></div><div><br></div><div>I use QEMU upstream master branch, updated today.</div><div><br></div><div>And my boot parameters are</div><div><br></div><div>BOOT_IMAGE=../vmlinuz-linux-vfio root=UUID=1c777330-3f00-4ab8-8d5d-d4b2b5d860b0 rw intel_iommu=on hugepages=2048 pci=assign-busses pcie_acs_override=downstream initrd=../initramfs-linux-vfio.img<br></div><div><div><br></div><div>Thanks to the ACS patched kernel, I'm able to successfully passthrough two different ixgbe physical functions (PF) to two different QEMU machines. Due to the IOMMU groups limitations of my hardware, that was not possible on the vanilla 4.10.8 kernel.</div><div><br></div><div>Once I create a VF from one of the PFs I have, I unbind that from the ixgbevf driver, and bind it to the vfio-pci driver. Then I use the following QEMU cmdline to do the PCI passthrough (the same command that I use to successfully passthrough PFs):</div><div><br></div><div># qemu-system-x86_64 /path/to/image.qcow2 -enable-kvm -smp 2 -m 2G -vga std -nographic -snapshot -device e1000,netdev=mgmt,mac=00:AA:BB:CC:0a:99 -netdev user,id=mgmt,hostfwd=tcp::20010-:22 -device vfio-pci,host=02:10.1<br></div><div><br></div><div>where 02:10.1 is the PCI identifier of the VF.</div><div><br></div><div>However, QEMU immediately crashes, reporting a full register dump:</div><div><br></div><div>******************************************************************************************************</div><div><div>qemu-system-x86_64: VFIO_MAP_DMA: -14</div><div>qemu-system-x86_64: vfio_dma_map(0x56017b4b75b0, 0xfebf0000, 0x4000, 0x7fcb60724000) = -14 (Bad address)</div><div>qemu: hardware error: vfio: DMA mapping failed, unable to continue</div><div>CPU #0:</div><div>EAX=00000103 EBX=80002004 ECX=00000103 EDX=00000cfc</div><div>ESI=00002000 EDI=00000000 EBP=7ffab670 ESP=00006d34</div><div>EIP=000eda90 EFL=00000046 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0</div><div>ES =0010 00000000 ffffffff 00c09300 DPL=0 DS [-WA]</div><div>CS =0008 00000000 ffffffff 00c09b00 DPL=0 CS32 [-RA]</div><div>SS =0010 00000000 ffffffff 00c09300 DPL=0 DS [-WA]</div><div>DS =0010 00000000 ffffffff 00c09300 DPL=0 DS [-WA]</div><div>FS =0010 00000000 ffffffff 00c09300 DPL=0 DS [-WA]</div><div>GS =0010 00000000 ffffffff 00c09300 DPL=0 DS [-WA]</div><div>LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT</div><div>TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS32-busy</div><div>GDT= 000f7070 00000037</div><div>IDT= 000f70ae 00000000</div><div>CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000</div><div>DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 </div><div>DR6=00000000ffff0ff0 DR7=0000000000000400</div><div>EFER=0000000000000000</div><div>FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80</div><div>FPR0=0000000000000000 0000 FPR1=0000000000000000 0000</div><div>FPR2=0000000000000000 0000 FPR3=0000000000000000 0000</div><div>FPR4=0000000000000000 0000 FPR5=0000000000000000 0000</div><div>FPR6=0000000000000000 0000 FPR7=0000000000000000 0000</div><div>XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000</div><div>XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000</div><div>XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000</div><div>XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000</div><div>CPU #1:</div><div>EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000663</div><div>ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000</div><div>EIP=0000fff0 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0</div><div>ES =0000 00000000 0000ffff 00009300</div><div>CS =f000 ffff0000 0000ffff 00009b00</div><div>SS =0000 00000000 0000ffff 00009300</div><div>DS =0000 00000000 0000ffff 00009300</div><div>FS =0000 00000000 0000ffff 00009300</div><div>GS =0000 00000000 0000ffff 00009300</div><div>LDT=0000 00000000 0000ffff 00008200</div><div>TR =0000 00000000 0000ffff 00008b00</div><div>GDT= 00000000 0000ffff</div><div>IDT= 00000000 0000ffff</div><div>CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000</div><div>DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 </div><div>DR6=00000000ffff0ff0 DR7=0000000000000400</div><div>EFER=0000000000000000</div><div>FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80</div><div>FPR0=0000000000000000 0000 FPR1=0000000000000000 0000</div><div>FPR2=0000000000000000 0000 FPR3=0000000000000000 0000</div><div>FPR4=0000000000000000 0000 FPR5=0000000000000000 0000</div><div>FPR6=0000000000000000 0000 FPR7=0000000000000000 0000</div><div>XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000</div><div>XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000</div><div>XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000</div><div>XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000</div><div>QEMU terminated with an exception</div></div><div>******************************************************************************************************<br></div><div><br></div><div>I've seen similar errors have been reported in various mailing lists and blog, but I was not able to relate them to my specific problem, nor find any solution.</div><div><br></div><div>Can anyone help?</div><div><br></div><div>Thanks,</div><div> Vincenzo</div><div><br></div><div><br></div>-- <br><div class="gmail_signature">Vincenzo Maffione<br></div> </div></div>