[vfio-users] Lost link when pass through rtl8168 to guest

Tue Sep 20 18:59:50 UTC 2016

Hi Wei,

My system is a desktop, so it must just be a general Gigabyte BIOS bug. I
submitted a help ticket about this issue and just gave a brief explanation
and then sent Alex's explanation. Hopefully it will be escalated correctly.

Thanks,
Nick

On Tue, Sep 20, 2016 at 1:08 PM, Wei Xu <wexu at redhat.com> wrote:

>
>
> On 2016年09月20日 22:20, Alex Williamson wrote:
>
>> On Tue, 20 Sep 2016 08:14:45 -0600
>> Alex Williamson <alex.williamson at redhat.com> wrote:
>>
>> On Tue, 20 Sep 2016 21:56:33 +0800
>>> Wei Xu <wexu at redhat.com> wrote:
>>>
>>> On 2016年09月20日 09:59, Alex Williamson wrote:
>>>>
>>>>> On Tue, 20 Sep 2016 09:28:57 +0800
>>>>> Wei Xu <wexu at redhat.com> wrote:
>>>>>
>>>>> Hi Guys,
>>>>>> I'm trying to pass through a rtl8168 nic to linux guest on my laptop
>>>>>> recently, the link is directly connected to my notebook with a cable.
>>>>>>
>>>>>> qemu: 2.7.0-rc4
>>>>>> host/guest kernel: 4.7.0-rc5
>>>>>> driver name: r8169
>>>>>>
>>>>>> After binding the driver to vfio-pci and launching the VM for a few
>>>>>> seconds, the connection led on the nic was turned off, and the guest
>>>>>> only see a link down nic with below messages.
>>>>>>
>>>>>> [    6.173188] r8169 0000:00:04.0 ens4: rtl_phy_reset_cond == 1 (loop:
>>>>>> 100, delay: 1).
>>>>>> [    6.177234] r8169 0000:00:04.0 ens4: link down
>>>>>> [    6.177592] r8169 0000:00:04.0 ens4: link down
>>>>>> [    6.177889] IPv6: ADDRCONF(NETDEV_UP): ens4: link is not ready
>>>>>>
>>>>>>
>>>>>> It's quite similar as below bug except it's for windows driver while
>>>>>> i'm testing linux.
>>>>>>
>>>>>> https://bugs.launchpad.net/qemu/+bug/1384892
>>>>>>
>>>>>>
>>>>>> More info:
>>>>>> My vm image is a pre-installed fedora 22 desktop, i also tried a fresh
>>>>>> fedora live iso, looks it can load the driver correctly.
>>>>>>
>>>>>> I tried to disable auto negotiation for the link but it didn't work
>>>>>> for me.
>>>>>>
>>>>>> I did the same test with my notebook with a Intel I218-LM ethernet
>>>>>> controller, it works pretty well every time.
>>>>>>
>>>>>> I googled around and looks it happened to bare metal too, so just
>>>>>> wonder
>>>>>> if this is a bug of network-manager, or is being caused by the nic
>>>>>> driver or an issue in qemu/kernel vfio, anybody can help?
>>>>>>
>>>>>
>>>>> Realtek nics don't work well with device assignment, they barely work
>>>>> well on bare metal.  Stick with the Intel nic or just use the rtl nic
>>>>> with virtio.  I've spent longer than I care to admit on the rtl quirks
>>>>> we have in QEMU and I expect they still only work with a few devices.
>>>>>
>>>>
>>>> OK, I'll ignore Realtek, so I added one Intel iwl6235 wireless nic to my
>>>> laptop, the pci tree shows both the rtl and iwl are behind a separate
>>>> pci bridge, after bind iwl to vfio-pci driver, i failed to pass through
>>>> it again with error message from qemu.
>>>>
>>>> qemu-system-x86_64: -device vfio-pci,host=0000:02:00.0: vfio: error,
>>>> group 5 is not viable, please ensure all devices within the iommu_group
>>>> are bound to their vfio bus driver.
>>>> qemu-system-x86_64: -device vfio-pci,host=0000:02:00.0: vfio: failed to
>>>> get group 5
>>>> qemu-system-x86_64: -device vfio-pci,host=0000:02:00.0: Device
>>>> initialization failed
>>>>
>>>> Seems it's caused by the rtl nic is under the same iommu group with iwl
>>>> as well, and when the kernel vfio driver checking the viablity, it'll
>>>> make sure all the devices under the same group are viable, it works fine
>>>> after i bound the rtl to vfio-pci too, i'm wonder if this a discipline?
>>>> can't i just bind the iwl nic and pass through the the guest?
>>>>
>>>> pci tree:
>>>> -[0000:00]-+-00.0 Intel Corporation Sky Lake Host Bridge/DRAM Registers
>>>> +-1c.0-[01]----00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168/8411
>>>> PCI Express Gigabit Ethernet Controller
>>>> +-1c.7-[02]----00.0 Intel Corporation Centrino Advanced-N 6235
>>>>
>>>
>>> If your PCH root ports report an ACS capability then you can run kernel
>>> v4.7 kernel on the host to expose the isolation.  If the root ports
>>> (00:1c.*) do not expose an ACS capability, it's probably a BIOS bug
>>> similar to Nick's system in this thread
>>> https://www.redhat.com/archives/vfio-users/2016-September/msg00059.html
>>>
>>
>> And I see you're running a v4.7 kernel already (though I'm not sure why
>> you're running an rc release for kernel or QEMU since both of those
>> have been released).  So you need to check them with lspci to see if an
>> ACS capability is exposed on the root ports, check whether your root
>> ports are covered by the device IDs in this quirk:
>>
>> http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.
>> git/commit/?id=1bf2bf229b64540f91ac6fa3af37c81249037a0b
>>
>> If there's no ACS capability but the root ports fall within the quirk,
>> it's a BIOS bug on the system.  Sorry.
>>
>
> Unfortunately, the device id is within your list in the commit qurik
> but it failed still, my ACS dump of pci is as the same as Nick's, just
> wondering why the bios doesn't report it, looks it's a default option
> for most of laptops, do you know what's the possible reason behind that?
> to connect all the components by default even with VT-d enabled?
>
> 00:1c.0 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root
> Port #5 (rev f1)
> 00: 86 80 14 a1 07 00 10 00 f1 00 04 06 00 00 81 00
> 10: 00 00 00 00 00 00 00 00 00 01 01 00 e0 e0 00 20
> 20: 10 df 10 df f1 ff 01 00 00 00 00 00 00 00 00 00
> 30: 00 00 00 00 40 00 00 00 00 00 00 00 0a 01 10 00
> 40: 10 80 42 01 01 80 00 00 00 00 10 00 13 40 72 05
> 50: 40 00 11 70 00 b2 44 00 00 00 40 01 00 00 00 00
> 60: 00 00 00 00 37 08 00 00 00 04 00 00 0e 00 00 00
> 70: 03 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
> 80: 05 90 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 90: 0d a0 00 00 58 14 01 50 00 00 00 00 00 00 00 00
> a0: 01 00 03 c8 00 00 00 00 00 00 00 00 00 00 00 00
> b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> d0: 01 10 00 07 42 18 00 00 08 00 1e 8b 00 00 00 00
> e0: 00 b7 f3 00 00 00 00 00 06 80 12 00 00 00 00 00
> f0: 50 00 00 00 00 03 00 40 b3 0f 33 08 00 00 00 01
> 100: 01 00 01 22 00 00 00 00 00 00 00 00 11 00 06 00
> 110: 00 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00
> 120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 140: 00 00 00 00 0f 00 00 00 00 00 00 00 00 00 00 00
> 150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 1a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 1b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 1c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 1d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 1e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 1f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 200: 00 00 00 00 1f 28 28 00 00 00 00 00 28 00 00 00
> 210: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 220: 19 00 01 00 00 00 00 00 00 00 00 00 77 75 77 75
>
>
> Thanks,
>
>>
>> Alex
>>
>>
> _______________________________________________
> vfio-users mailing list
> vfio-users at redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160920/81285334/attachment.htm>