[vfio-users] fitlet I211 PCI passthrough
globalgorrilla at fastmail.fm
globalgorrilla at fastmail.fm
Sat Mar 26 15:25:05 UTC 2016
Even while I use SR-IOV interfaces passed through, I still find
openvswitch very useful.
In your scenario I might just use openvswitch and bind all the NICs.
Perhaps use DPDK if you're primarily doing networking with the device.
Also CoreOS might be nice if again you're building a dedicated
router/firewall.
On 26 Mar 2016, at 4:28, YAEGASHI Takeshi wrote:
> Hello,
>
> I have a fitlet http://www.fit-pc.com/web/products/fitlet/ with 4 GbE
> (I211) ports and wanted to passthrough some of them to KVM guests, ie
> assign one I211 to each guests. I've tried various configurations
> with libvirt/kvm but no luck so far.
>
> After reading
> http://vfio.blogspot.jp/2014/08/iommu-groups-inside-and-out.html
> I've got sure that there's a problem in IOMMU groups. Actually all of
> I211 and AMD's PCI bridge [1022:156b] share the same iommu_group 2.
>
> $ lspci -tv
> -[0000:00]-+-00.0 Advanced Micro Devices, Inc. [AMD] Device 1566
> +-00.2 Advanced Micro Devices, Inc. [AMD] Device 1567
> +-01.0 Advanced Micro Devices, Inc. [AMD/ATI] Mullins
> [Radeon R6 Graphics]
> +-01.1 Advanced Micro Devices, Inc. [AMD/ATI] Kabini
> HDMI/DP Audio
> +-02.0 Advanced Micro Devices, Inc. [AMD] Device 156b
> +-02.2-[01]----00.0 Intel Corporation I211 Gigabit Network
> Connection
> +-02.3-[02]----00.0 Intel Corporation I211 Gigabit Network
> Connection
> +-02.4-[03]----00.0 Intel Corporation I211 Gigabit Network
> Connection
> +-02.5-[04]----00.0 Intel Corporation I211 Gigabit Network
> Connection
> +-08.0 Advanced Micro Devices, Inc. [AMD] Device 1537
> +-10.0 Advanced Micro Devices, Inc. [AMD] FCH USB XHCI
> Controller
> +-11.0 Advanced Micro Devices, Inc. [AMD] FCH SATA
> Controller [AHCI mode]
> +-12.0 Advanced Micro Devices, Inc. [AMD] FCH USB EHCI
> Controller
> +-13.0 Advanced Micro Devices, Inc. [AMD] FCH USB EHCI
> Controller
> +-14.0 Advanced Micro Devices, Inc. [AMD] FCH SMBus
> Controller
> +-14.2 Advanced Micro Devices, Inc. [AMD] FCH Azalia
> Controller
> +-14.3 Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge
> +-14.7 Advanced Micro Devices, Inc. [AMD] FCH SD Flash
> Controller
> +-18.0 Advanced Micro Devices, Inc. [AMD] Device 1580
> +-18.1 Advanced Micro Devices, Inc. [AMD] Device 1581
> +-18.2 Advanced Micro Devices, Inc. [AMD] Device 1582
> +-18.3 Advanced Micro Devices, Inc. [AMD] Device 1583
> +-18.4 Advanced Micro Devices, Inc. [AMD] Device 1584
> \-18.5 Advanced Micro Devices, Inc. [AMD] Device 1585
>
> $ find /sys/kernel/iommu_groups -type l
> /sys/kernel/iommu_groups/0/devices/0000:00:00.0
> /sys/kernel/iommu_groups/1/devices/0000:00:01.0
> /sys/kernel/iommu_groups/1/devices/0000:00:01.1
> /sys/kernel/iommu_groups/2/devices/0000:00:02.0
> /sys/kernel/iommu_groups/2/devices/0000:00:02.2
> /sys/kernel/iommu_groups/2/devices/0000:00:02.3
> /sys/kernel/iommu_groups/2/devices/0000:00:02.4
> /sys/kernel/iommu_groups/2/devices/0000:00:02.5
> /sys/kernel/iommu_groups/2/devices/0000:01:00.0
> /sys/kernel/iommu_groups/2/devices/0000:02:00.0
> /sys/kernel/iommu_groups/2/devices/0000:03:00.0
> /sys/kernel/iommu_groups/2/devices/0000:04:00.0
> /sys/kernel/iommu_groups/3/devices/0000:00:08.0
> /sys/kernel/iommu_groups/4/devices/0000:00:10.0
> /sys/kernel/iommu_groups/5/devices/0000:00:11.0
> /sys/kernel/iommu_groups/6/devices/0000:00:12.0
> /sys/kernel/iommu_groups/7/devices/0000:00:13.0
> /sys/kernel/iommu_groups/8/devices/0000:00:14.0
> /sys/kernel/iommu_groups/8/devices/0000:00:14.2
> /sys/kernel/iommu_groups/8/devices/0000:00:14.3
> /sys/kernel/iommu_groups/8/devices/0000:00:14.7
> /sys/kernel/iommu_groups/9/devices/0000:00:18.0
> /sys/kernel/iommu_groups/9/devices/0000:00:18.1
> /sys/kernel/iommu_groups/9/devices/0000:00:18.2
> /sys/kernel/iommu_groups/9/devices/0000:00:18.3
> /sys/kernel/iommu_groups/9/devices/0000:00:18.4
> /sys/kernel/iommu_groups/9/devices/0000:00:18.5
>
> I'm running Ubuntu 14.04 with xenial kernel 4.4.0-13, libvirt 1.2.2,
> qemu 2.0.0. Using vfio-pci simply failed:
>
> qemu-system-x86_64: -device
> vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x3: vfio: error,
> group 2 is not viable, please ensure all devices within the
> iommu_group are bound to their vfio bus driver.
> qemu-system-x86_64: -device
> vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x3: vfio: failed to
> get group 2
> qemu-system-x86_64: -device
> vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x3: Device
> initialization failed.
> qemu-system-x86_64: -device
> vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x3: Device
> 'vfio-pci' could not be initialized
>
> Using legacy KVM device assignment with <driver name='kvm'/> also
> failed with unclear reason "Invalid argument":
>
> qemu-system-x86_64: -device
> pci-assign,configfd=24,host=01:00.0,id=hostdev0,bus=pci.2,addr=0x2:
> Failed to assign device "hostdev0" : Invalid argument
> qemu-system-x86_64: -device
> pci-assign,configfd=24,host=01:00.0,id=hostdev0,bus=pci.2,addr=0x2:
> Device initialization failed.
> qemu-system-x86_64: -device
> pci-assign,configfd=24,host=01:00.0,id=hostdev0,bus=pci.2,addr=0x2:
> Device 'kvm-pci-assign' could not be initialized
>
> After some googling I've learned that legacy KVM assignment also
> refuses to work for this configuration since kernel 4.2
> https://lkml.org/lkml/2015/11/12/661 . With older kernel 3.19 this
> configuration worked as expected.
>
> Is there any chance for me to achieve single I211 passthrough with
> recent kernels? Because the hardware has no flexibility on the bus
> topologies, the only possible way would be patching the kernel (ACS
> override or PCI quirks). Is it safe? Does anyone have any info on
> the IOMMU capability of device/bridge/chipset in question?
>
> Regards,
> --
> YAEGASHI Takeshi <yaegashi at debian.org>
>
> _______________________________________________
> vfio-users mailing list
> vfio-users at redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
More information about the vfio-users
mailing list