[vfio-users] VFIO and KSM (and maybe hugepages)

[Alex Williamson]

On Tue, 31 May 2016 20:20:58 +0000
Colin Godsey <crgodsey at gmail.com> wrote:

> I had a few questions regarding general ‘page management’ and VFIO, mostly
> related to kernel shared pages.
> 
> I have a host running 2 virtual ‘gaming rigs’ with a single dedicated GPU
> each. I had an intermittent problem where when gaming (on the same game)
> with both rigs, one would receive graphic artifacts. Specifically I would
> see triangle/geometry artifacts which usually indicate corrupt GPU RAM.
> 
> Both cards are so completely different, and different generation, one is
> really new, I didn’t believe it was bad VRAM. Graphics drivers to swap
> various buffers from system RAM to VRAM so I figured it could also be
> something related to system RAM.
> 
> I disabled any kind of… alternative page management I could- swap, KSM,
> huge pages etc. and it did fix it. Because the issue only would effect one
> machine, and I only observed it when the same game was running on both, I
> assumed maybe it was related to KSM.
> 
> *Is there any possible way KSM could interfere with the DMAR in some way
> where it tries to share/alter DMA regions?* And broader: what prevents
> systems like khugepage, kswap, and ksm from interfering with these regions
> in the first place? I’ve read that transparent hugepages can interfere with
> VFIO, is it safe to assume that other DMA issues could arise with other
> types of page management?

What kernel were you running where you saw this?  vfio uses
get_user_pages to increase the reference count on pages mapped through
the iommu.  This should prevent both ksm and transparent hugepages from
being able to operate on the pages.  Kernel v4.5 had a bug (now fixed
in v4.5.5) that did not honor the reference, allowing thp (maybe ksm
too) to still operate on those pages.  So as long as you're not running
v4.5.0 through v4.5.4 (or a v4.5-rc), I'm not aware of any issues with
page pinning.  Thanks,

Alex