[vfio-users] ACS and IOMMU groups on Skylake/Sunrise

[Alex Williamson]

On Wed, 18 Apr 2018 17:20:51 -0400
Philip Kania <philip.kania at onshape.com> wrote:

> I'm trying to setup vfio-pci for a kvm guest. I am running
> 4.13.0-38-generic. I have a I350 NIC chipset and i7-6700K CPU.
> drivers/pci/quirks.c for my kernel version\
> 
>  appears to have support for the I350 and the Skylake/Sunrise Point PCIe
> ACS register issues.
> 
> 
> dmesg shows DMAR: Intel(R) Virtualization Technology for Directed I/O
> 
> 
> and lspci shows the virtual function(s) on my NIC, but all the VF and PF
> along with PCI bridges all appear in the same group.
> 
> 
> IOMMU Group 6 00:1c.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H
> PCI Express Root Port #3 [8086:a112] (rev f1)
> 
> IOMMU Group 6 00:1c.4 PCI bridge [0604]: Intel Corporation Sunrise Point-H
> PCI Express Root Port #5 [8086:a114] (rev f1)
> 
> IOMMU Group 6 07:00.0 PCI bridge [0604]: ASMedia Technology Inc.
> ASM1083/1085 PCIe to PCI Bridge [1b21:1080] (rev 04)
> 
> IOMMU Group 6 09:00.0 Ethernet controller [0200]: Intel Corporation I350
> Gigabit Network Connection [8086:1521] (rev 01)
> 
> IOMMU Group 6 09:00.1 Ethernet controller [0200]: Intel Corporation I350
> Gigabit Network Connection [8086:1521] (rev 01)
> 
> IOMMU Group 6 09:10.0 Ethernet controller [0200]: Intel Corporation I350
> Ethernet Controller Virtual Function [8086:1520] (rev 01)
> 
> 
> I expected that, with the quirks, by VF would appear in its own group. Any
> idea what I'm missing?

Can't really tell what your hierarchy is here, are you suggesting you
have an ASMedia PCI bridge sitting between your root port and the
I350?  That seems unlikely, although it'd explain the grouping.  Please
provide full 'sudo lspci -vvv' output and
'find /sys/kernel/iommu_groups/ -type l'.  Thanks,

Alex