[vfio-users] AMD Ryzen Nested Page Table Performance Oddities

[Nick Sarnie]

On Wed, Apr 5, 2017 at 11:32 PM, Nick Sarnie <commendsarnex at gmail.com> wrote:
> Hi all,
>
> I'm seeing some strange issues regarding the Nested Page Table option
> of the kvm-amd module on my Ryzen system.
>
> First off, having the parameter enabled, as it is by default, clearly
> and consistently increases CPU performance inside the VM. I've tested
> this with a kernel compile, and it is around 5 times longer with npt
> disabled using tmpfs. Passing any PCI device through to the VM does
> not affect the CPU performance.
>
> Now, if I pass through my graphics card, we see some very strange
> issues. If npt is disabled, the FPS in 3D applications is between 2

    Sorry, I mean 'if npt is enabled'.

> and 4 times lower than if npt is disabled. This is a consistent,
> cross-VM-platform result. In Windows 10, Dota 2 runs at around 35 FPS
> with npt enabled, and around 110 with npt disabled. In Fedora 25,
> Heaven benchmark results in an average of 32 FPS with npt enabled, and
> 48 with npt disabled. I haven't found a 3D application where there is
> not a significant GPU performance hit to enabling npt.
>
> I would like to try to understand this issue, as anyone wanting to use
> a GPU Passthrough config on Ryzen basically is forced to disable npt
> and suffer a CPU performance hit.
>
> I can provide any performance statistics, logs, or test any patches.
>
> Thank you,
> Sarnex