[vfio-users] Huge performance decrease in VM

[Thomas Lindroth]

On 01/06/2017 01:00 PM, Marius Steffen wrote:
> Is there anything I can do to a) find the cause of this bad performance
> b) make my VM perform better?

One thing you can try is to run the game with poor performance and at the
same time run "perf kvm --host stat live" on the host. This will show how
many VM-EXIT kvm performs. VM-EXIT means the guest has done something the
hardware virtualisation can't handle, like accessing MSR or virtual IO.

Some workloads will perform badly because they cause a lot of VM-EXIT.
Perf will tell you why a VM-EXIT was performed. The more common causes are
IO_INSTRUCTION and EPT_MISCONFIG. IO_INSTRUCTION means the guest tried to
access an x86 ioport and EPT_MISCONFIG seems to be an odd name for access
to memory mapped io. Sometimes a lot of time is spent in HLT but that's
normal. It only means the guest OS has nothing to do and goes idle.

By running "perf kvm --host stat live --event=ioport" you can check which
ioport is accessed and "perf kvm --host stat live --event=mmio" shows
which address was accessed for memory mapped io.

If you see that an mmio address is accessed a lot and want to find out what
it is you can run "virsh qemu-monitor-command <nameofvm> --hmp 'info mtree'"
to get a list of the memory layout of the VM. Unfortunately I don't know
of any way to get a similar list of ioports.

As an example on my system some games like Rise of the Tomb Raider and
Assassin's Creed Unity will access ioport 0xb008 more than 200,000 times/sec
and those games will have poor performance compared to native. Qemu
hardcode the acpi timer to ioport 0xb008 and those games excessively read
the hardware timer. I haven't figured out any way to work around the problem
but I run games with vsync on and even with the decreased performance I get
60fps in those titles.