[vfio-users] Lost link when pass through rtl8168 to guest

[Alex Williamson]

On Tue, 20 Sep 2016 21:56:33 +0800
Wei Xu <wexu at redhat.com> wrote:

> On 2016年09月20日 09:59, Alex Williamson wrote:
> > On Tue, 20 Sep 2016 09:28:57 +0800
> > Wei Xu <wexu at redhat.com> wrote:
> >  
> >> Hi Guys,
> >> I'm trying to pass through a rtl8168 nic to linux guest on my laptop
> >> recently, the link is directly connected to my notebook with a cable.
> >>
> >> qemu: 2.7.0-rc4
> >> host/guest kernel: 4.7.0-rc5
> >> driver name: r8169
> >>
> >> After binding the driver to vfio-pci and launching the VM for a few
> >> seconds, the connection led on the nic was turned off, and the guest
> >> only see a link down nic with below messages.
> >>
> >> [    6.173188] r8169 0000:00:04.0 ens4: rtl_phy_reset_cond == 1 (loop:
> >> 100, delay: 1).
> >> [    6.177234] r8169 0000:00:04.0 ens4: link down
> >> [    6.177592] r8169 0000:00:04.0 ens4: link down
> >> [    6.177889] IPv6: ADDRCONF(NETDEV_UP): ens4: link is not ready
> >>
> >>
> >> It's quite similar as below bug except it's for windows driver while
> >> i'm testing linux.
> >>
> >> https://bugs.launchpad.net/qemu/+bug/1384892
> >>
> >>
> >> More info:
> >> My vm image is a pre-installed fedora 22 desktop, i also tried a fresh
> >> fedora live iso, looks it can load the driver correctly.
> >>
> >> I tried to disable auto negotiation for the link but it didn't work for me.
> >>
> >> I did the same test with my notebook with a Intel I218-LM ethernet
> >> controller, it works pretty well every time.
> >>
> >> I googled around and looks it happened to bare metal too, so just wonder
> >> if this is a bug of network-manager, or is being caused by the nic
> >> driver or an issue in qemu/kernel vfio, anybody can help?  
> >
> > Realtek nics don't work well with device assignment, they barely work
> > well on bare metal.  Stick with the Intel nic or just use the rtl nic
> > with virtio.  I've spent longer than I care to admit on the rtl quirks
> > we have in QEMU and I expect they still only work with a few devices.  
> 
> OK, I'll ignore Realtek, so I added one Intel iwl6235 wireless nic to my 
> laptop, the pci tree shows both the rtl and iwl are behind a separate 
> pci bridge, after bind iwl to vfio-pci driver, i failed to pass through 
> it again with error message from qemu.
> 
> qemu-system-x86_64: -device vfio-pci,host=0000:02:00.0: vfio: error, 
> group 5 is not viable, please ensure all devices within the iommu_group 
> are bound to their vfio bus driver.
> qemu-system-x86_64: -device vfio-pci,host=0000:02:00.0: vfio: failed to 
> get group 5
> qemu-system-x86_64: -device vfio-pci,host=0000:02:00.0: Device 
> initialization failed
> 
> Seems it's caused by the rtl nic is under the same iommu group with iwl 
> as well, and when the kernel vfio driver checking the viablity, it'll 
> make sure all the devices under the same group are viable, it works fine 
> after i bound the rtl to vfio-pci too, i'm wonder if this a discipline? 
> can't i just bind the iwl nic and pass through the the guest?
> 
> pci tree:
> -[0000:00]-+-00.0 Intel Corporation Sky Lake Host Bridge/DRAM Registers
> +-1c.0-[01]----00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 
> PCI Express Gigabit Ethernet Controller
> +-1c.7-[02]----00.0 Intel Corporation Centrino Advanced-N 6235

If your PCH root ports report an ACS capability then you can run kernel
v4.7 kernel on the host to expose the isolation.  If the root ports
(00:1c.*) do not expose an ACS capability, it's probably a BIOS bug
similar to Nick's system in this thread
https://www.redhat.com/archives/vfio-users/2016-September/msg00059.html
Thanks,

Alex