<html> <head> <meta content="text/html; charset=utf-8" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000"> Dear Mr. Williamson, Proxmox ve 4.1 has your ACS-override-patch included in the kernel. When "pcie_acs_override=downstream" added to kernel parameter, dmesg says "PCIe ACS overrides enabled; ..." However, iommu group 4 is still big. I have patched the pve-kernel with your Skylake-PCH-patch. It works now. IOMMU groups are separated. Each pcie device is in its own group. Thank you very much. <div class="moz-cite-prefix">On 4/5/2016 9:12 PM, Alex Williamson wrote: </div> <blockquote cite="mid:CAEMbtc+L8c2Ss0gUJxYFqGsi5cpaTr0fKZZZYLJJPucyezGU4A@mail.gmail.com" type="cite"> <div dir="ltr"> <div class="gmail_extra"> <div class="gmail_quote">On Tue, Apr 5, 2016 at 3:13 AM, Tran The Anh <<a moz-do-not-send="true" href="mailto:anh.tranthe@thegioididong.com" target="_blank">anh.tranthe@thegioididong.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> <div bgcolor="#FFFFFF" text="#000000"> Hi, I would like to passthrough some intel i210 ports to KVM's VM, but has encountered the iommu group issue: <blockquote>/sys/kernel/iommu_groups/0/devices/0000:00:00.0 /sys/kernel/iommu_groups/1/devices/0000:00:14.0 <= usb /sys/kernel/iommu_groups/1/devices/0000:00:14.2 /sys/kernel/iommu_groups/2/devices/0000:00:16.0 /sys/kernel/iommu_groups/2/devices/0000:00:16.1 /sys/kernel/iommu_groups/3/devices/0000:00:17.0 <= sata /sys/kernel/iommu_groups/4/devices/0000:00:1c.0 <= pch port /sys/kernel/iommu_groups/4/devices/0000:00:1c.1 <= pch port /sys/kernel/iommu_groups/4/devices/0000:00:1c.2 <= pch port /sys/kernel/iommu_groups/4/devices/0000:00:1c.3 <= pch port /sys/kernel/iommu_groups/4/devices/0000:00:1c.6 <= pch port /sys/kernel/iommu_groups/4/devices/0000:01:00.0 <= i210 /sys/kernel/iommu_groups/4/devices/0000:02:00.0 <= i210 /sys/kernel/iommu_groups/4/devices/0000:03:00.0 <= i210 /sys/kernel/iommu_groups/4/devices/0000:04:00.0 <= i210 /sys/kernel/iommu_groups/4/devices/0000:05:00.0 /sys/kernel/iommu_groups/4/devices/0000:06:00.0 <= vga aspeed 2400 /sys/kernel/iommu_groups/5/devices/0000:00:1f.0 /sys/kernel/iommu_groups/5/devices/0000:00:1f.2 /sys/kernel/iommu_groups/5/devices/0000:00:1f.4 </blockquote> My system is: <blockquote>Intel Xeon e3-1220 v5 Supermicro x11ssh-ln4f </blockquote> According to board's diagram: <a moz-do-not-send="true" href="http://www.supermicro.com/manuals/motherboard/C236/MNL-1778.pdf" target="_blank">http://www.supermicro.com/manuals/motherboard/C236/MNL-1778.pdf</a> Those i210s are connected to PCH. It seems that PCH ports have ACS capabilities: <blockquote>lspci -vvvnk | egrep -i 'Access|Rev' 00:17.0 0106: 8086:a102 (rev 31) (prog-if 01 [AHCI 1.0]) 00:1c.0 0604: 8086:a110 (rev f1) (prog-if 00 [Normal decode]) Capabilities: [140 v1] Access Control Services 00:1c.1 0604: 8086:a111 (rev f1) (prog-if 00 [Normal decode]) Capabilities: [140 v1] Access Control Services 00:1c.2 0604: 8086:a112 (rev f1) (prog-if 00 [Normal decode]) Capabilities: [140 v1] Access Control Services 00:1c.3 0604: 8086:a113 (rev f1) (prog-if 00 [Normal decode]) Capabilities: [140 v1] Access Control Services 00:1c.6 0604: 8086:a116 (rev f1) (prog-if 00 [Normal decode]) Capabilities: [140 v1] Access Control Services </blockquote> I have tried [proxmox ve 4.0, proxmox 4.1, centos 7, ubuntu 1404, ubuntu 1510] + add "pci_acs_override=downstream" to kernel parameters. But the iommu group 4 is always the same. </div> </blockquote> <div> </div> <div>The ACS override patch is not upstream, you'd actually need to run a kernel with support for it for the kernel command line option to work. You have a Sunrise Point PCH where Intel has a broken implementation of ACS. I've posted quirks[1] for this, but they probably won't be available until a v4.7 kernel. Your choices are to either patch your kernel with the ACS override patch or patch your kernel with the quirks (preferred) until support is merged into the kernel. Thanks,</div> <div> </div> <div>Alex</div> <div> </div> <div>[1] <a moz-do-not-send="true" href="https://lkml.org/lkml/2016/3/31/1112">https://lkml.org/lkml/2016/3/31/1112</a></div> </div> </div> </div> </blockquote> <table style="border-top: 1px solid #aaabb6;"> <tr> <td style="width: 55px; padding-top: 13px;"><a href="https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient&utm_term=icon" target="_blank"><img src="https://ipmcdn.avast.com/images/2016/icons/icon-envelope-open-tick-round-orange-v1.png" /></a></td> <td style="width: 470px; padding-top: 15px; color: #41424e; font-size: 13px; font-family: Arial, Helvetica, sans-serif; line-height: 18px;">Virus-free. <a href="https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient&utm_term=link" target="_blank" style="color: #4453ea;">www.avast.com</a> </td> </tr> </table></body> </html>