<div dir="ltr"><div><div><div>Hi Wei, </div>My system is a desktop, so it must just be a general Gigabyte BIOS bug. I submitted a help ticket about this issue and just gave a brief explanation and then sent Alex's explanation. Hopefully it will be escalated correctly. </div>Thanks, </div>Nick </div><div class="gmail_extra"> <div class="gmail_quote">On Tue, Sep 20, 2016 at 1:08 PM, Wei Xu <<a href="mailto:wexu@redhat.com" target="_blank">wexu@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5"> On 2016年09月20日 22:20, Alex Williamson wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> On Tue, 20 Sep 2016 08:14:45 -0600 Alex Williamson <<a href="mailto:alex.williamson@redhat.com" target="_blank">alex.williamson@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> On Tue, 20 Sep 2016 21:56:33 +0800 Wei Xu <<a href="mailto:wexu@redhat.com" target="_blank">wexu@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> On 2016年09月20日 09:59, Alex Williamson wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> On Tue, 20 Sep 2016 09:28:57 +0800 Wei Xu <<a href="mailto:wexu@redhat.com" target="_blank">wexu@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Hi Guys, I'm trying to pass through a rtl8168 nic to linux guest on my laptop recently, the link is directly connected to my notebook with a cable. qemu: 2.7.0-rc4 host/guest kernel: 4.7.0-rc5 driver name: r8169 After binding the driver to vfio-pci and launching the VM for a few seconds, the connection led on the nic was turned off, and the guest only see a link down nic with below messages. [ 6.173188] r8169 0000:00:04.0 ens4: rtl_phy_reset_cond == 1 (loop: 100, delay: 1). [ 6.177234] r8169 0000:00:04.0 ens4: link down [ 6.177592] r8169 0000:00:04.0 ens4: link down [ 6.177889] IPv6: ADDRCONF(NETDEV_UP): ens4: link is not ready It's quite similar as below bug except it's for windows driver while i'm testing linux. <a href="https://bugs.launchpad.net/qemu/+bug/1384892" rel="noreferrer" target="_blank">https://bugs.launchpad.net/qemu/+bug/1384892</a> More info: My vm image is a pre-installed fedora 22 desktop, i also tried a fresh fedora live iso, looks it can load the driver correctly. I tried to disable auto negotiation for the link but it didn't work for me. I did the same test with my notebook with a Intel I218-LM ethernet controller, it works pretty well every time. I googled around and looks it happened to bare metal too, so just wonder if this is a bug of network-manager, or is being caused by the nic driver or an issue in qemu/kernel vfio, anybody can help? </blockquote> Realtek nics don't work well with device assignment, they barely work well on bare metal. Stick with the Intel nic or just use the rtl nic with virtio. I've spent longer than I care to admit on the rtl quirks we have in QEMU and I expect they still only work with a few devices. </blockquote> OK, I'll ignore Realtek, so I added one Intel iwl6235 wireless nic to my laptop, the pci tree shows both the rtl and iwl are behind a separate pci bridge, after bind iwl to vfio-pci driver, i failed to pass through it again with error message from qemu. qemu-system-x86_64: -device vfio-pci,host=0000:02:00.0: vfio: error, group 5 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver. qemu-system-x86_64: -device vfio-pci,host=0000:02:00.0: vfio: failed to get group 5 qemu-system-x86_64: -device vfio-pci,host=0000:02:00.0: Device initialization failed Seems it's caused by the rtl nic is under the same iommu group with iwl as well, and when the kernel vfio driver checking the viablity, it'll make sure all the devices under the same group are viable, it works fine after i bound the rtl to vfio-pci too, i'm wonder if this a discipline? can't i just bind the iwl nic and pass through the the guest? pci tree: -[0000:00]-+-00.0 Intel Corporation Sky Lake Host Bridge/DRAM Registers +-1c.0-[01]----00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller +-1c.7-[02]----00.0 Intel Corporation Centrino Advanced-N 6235 </blockquote> If your PCH root ports report an ACS capability then you can run kernel v4.7 kernel on the host to expose the isolation. If the root ports (00:1c.*) do not expose an ACS capability, it's probably a BIOS bug similar to Nick's system in this thread <a href="https://www.redhat.com/archives/vfio-users/2016-September/msg00059.html" rel="noreferrer" target="_blank">https://www.redhat.com/archives/vfio-users/2016-September/msg00059.html</a> </blockquote> And I see you're running a v4.7 kernel already (though I'm not sure why you're running an rc release for kernel or QEMU since both of those have been released). So you need to check them with lspci to see if an ACS capability is exposed on the root ports, check whether your root ports are covered by the device IDs in this quirk: <a href="http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=1bf2bf229b64540f91ac6fa3af37c81249037a0b" rel="noreferrer" target="_blank">http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=1bf2bf229b64540f91ac6fa3af37c81249037a0b</a> If there's no ACS capability but the root ports fall within the quirk, it's a BIOS bug on the system. Sorry. </blockquote> </div></div> Unfortunately, the device id is within your list in the commit qurik but it failed still, my ACS dump of pci is as the same as Nick's, just wondering why the bios doesn't report it, looks it's a default option for most of laptops, do you know what's the possible reason behind that? to connect all the components by default even with VT-d enabled? 00:1c.0 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root Port #5 (rev f1) 00: 86 80 14 a1 07 00 10 00 f1 00 04 06 00 00 81 00 10: 00 00 00 00 00 00 00 00 00 01 01 00 e0 e0 00 20 20: 10 df 10 df f1 ff 01 00 00 00 00 00 00 00 00 00 30: 00 00 00 00 40 00 00 00 00 00 00 00 0a 01 10 00 40: 10 80 42 01 01 80 00 00 00 00 10 00 13 40 72 05 50: 40 00 11 70 00 b2 44 00 00 00 40 01 00 00 00 00 60: 00 00 00 00 37 08 00 00 00 04 00 00 0e 00 00 00 70: 03 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 80: 05 90 00 00 00 00 00 00 00 00 00 00 00 00 00 00 90: 0d a0 00 00 58 14 01 50 00 00 00 00 00 00 00 00 a0: 01 00 03 c8 00 00 00 00 00 00 00 00 00 00 00 00 b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 d0: 01 10 00 07 42 18 00 00 08 00 1e 8b 00 00 00 00 e0: 00 b7 f3 00 00 00 00 00 06 80 12 00 00 00 00 00 f0: 50 00 00 00 00 03 00 40 b3 0f 33 08 00 00 00 01 100: 01 00 01 22 00 00 00 00 00 00 00 00 11 00 06 00 110: 00 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 140: 00 00 00 00 0f 00 00 00 00 00 00 00 00 00 00 00 150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 1a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 1b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 1c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 1d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 1e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 1f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 200: 00 00 00 00 1f 28 28 00 00 00 00 00 28 00 00 00 210: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 220: 19 00 01 00 00 00 00 00 00 00 00 00 77 75 77 75<div class="HOEnZb"><div class="h5"> Thanks, <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Alex </blockquote> _______________________________________________ vfio-users mailing list <a href="mailto:vfio-users@redhat.com" target="_blank">vfio-users@redhat.com</a> <a href="https://www.redhat.com/mailman/listinfo/vfio-users" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/vfio-users</a> </div></div></blockquote></div> </div>