[vfio-users] Linux iommu issue with peer-to-peer dma transfers between NVidia GTX 1080s

[Zir Blazer]

I recall that there were some issues (Not necessarily yours) with P2P in nVidia cards related to ACS. While anyone that cares about IOMMU Groups will love ACS due to the better granularity, it broke what you want to use, and you may want to check if you can disable it. Check these:


https://www.supermicro.com/support/faqs/faq.cfm?faq=20732
https://www.supermicro.com/support/faqs/faq.cfm?faq=20264

Basically, the Processor Root Ports in Intel HEDT platform supports ACS, and my uneducated guess is that when you enable the IOMMU, Linux also enables their ACS capabilities. I have no idea if setpci can be used to disable ACS in the reelevant Root Ports so P2P can resume as usual, nor how to use it. From the IOMMU Groups perspective, I suppose that they will be in the same group. May be worth a try...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20170924/f3c93c4d/attachment.htm>