[vfio-users] threadripper slowness

Thu Aug 15 02:49:07 UTC 2019

You need to tweak NUMA because your previous CPU was not NUMA, and your new
one is. Having the VM running on some CPU cores on one physical package,
but accessing ram that's attached to other CPU cores on a different package
can cause a lot of overhead. You want to avoid that.

On Wed, Aug 14, 2019 at 7:31 PM Roger Lawhorn <rll at twc.com> wrote:

> Just a note:
> So far to resolve this I have had to use the kernel boot line options
> "rcu_nocbs=0-23 processor.max_cstate=5".
> This fixes a known hardware bug in ryzen cpus that leads to constant
> crashing in linux.
> I also compiled qemu 4.1.0-rc4.
> Though it seems better I still suffer severe lag during heavy loading of
> textures and 3d objects.
> The test games are fallout 4 and fallout 76.
> The virtual machine is stored on a 1tb intel ssd.
> On a 4 core i7 4940mx it was perfect.
> On my new 12 core threadripper it lags out.
> As long as the game isnt loading anything new it runs fine.
> I never tweaked 'numa' before and I don't see why i need to now.
>
> Any ideas?
>
>
> I am on this till it works or I am out $3000.
>
>
> On 8/12/19 6:56 AM, Roger Lawhorn wrote:
>
>
>
> On 8/11/19 9:59 PM, gamanakis at gmail.com wrote:
>
> You could probably get rid of all the lag you experience. Possibly qemu 4.0
> provides better topology to the guest. Generally Threadripper CPUs have 2
> NUMA nodes, and you would want to pin the vcpus of the guest to one NUMA
> node, and also allocate RAM from the same NUMA node. You would also have to
> make sure the passthrough devices sit on corresponding PCIe lanes which that
> NUMA node controls. There is lots of good info in reddit.com/r/vfio. Please
> also provide your xml file (or script), and what you use as a virtualized
> hard drive. Is it a passthrough NVMe, or virtio-scsi? In the latter case you
> should also pin iothreads on the same NUMA node as everything else.
> You can see the topology and NUMA nodes of your cpu by running lstopo.
>
> I am eager to see a configuration that uses both NUMA nodes of a
> Threadripper CPU in one guest with minimal latency.
>
>
> -----Original Message-----
> From: vfio-users-bounces at redhat.com <vfio-users-bounces at redhat.com> <vfio-users-bounces at redhat.com>
> On Behalf Of Roger Lawhorn
> Sent: Sunday, August 11, 2019 9:31 PM
> To: vfio-users at redhat.com
> Subject: Re: [vfio-users] threadripper slowness
>
> Hello,
>
> Just wanted to mention that I got rid of a lot of the lag by compiling
>
> qemu 4.0.
>
> I dod not know if its a threadripper issue or not.
> The lag directly coincides with hard drive access.
> No hard drive access, no game lag.
> I have redhat virtio drivers installed for the hard drive and the nic.
> Are these drivers intel only?
> They are left only from my i7-4940mx processor install.
>
>
> On 8/9/19 11:51 AM, Roger Lawhorn wrote:
>
> Hello.
> I am new to the list.
> I have been doing gpu passthrough for almost 5 years now with a i7
> 4940mx cpu on a msi gt70 laptop.
>
> My new PC build is:
> MSI x399 carbon pro motherboard : bios xxxxxx.1c0 : svm=enabled:
> iommu=enabled
> Threadripper 2920 12 core
> Radeon pro duo R9 Fury X
> Nvidia 980 TI OC
>
> I was unable to passthrough the nvidia card due to it not getting past
> the "Running option rom..." message when debugging.
>
> The Radeon passed through with flying colors.
>
> The issue:
> I used the radeon to run triple A games in linux using steam for linux
> and proton. No problems.
>
> In Windows 7 in qemu I have passed all 24 cpus to it.
> They max out to 80% while playing the same triple A games and the
> games lag.
> I have never had this issue before.
>
> Admittedly I am using the same script I used on the i7 to get going.
> I had to remove threads from the -smp switch and use cores only.
>
> I also removed the kvm=off which is used for nvidia cards.
>
> I am looking for links to articles or direct info on what to do to
> optimize the cpus.
> When not lagging the radeon performs extremely well.
>
> I am concerned that perhaps kvm is not being used.
>
>
> Thanks
>
> _______________________________________________
> vfio-users mailing listvfio-users at redhat.comhttps://www.redhat.com/mailman/listinfo/vfio-users
>
> _______________________________________________
> vfio-users mailing listvfio-users at redhat.comhttps://www.redhat.com/mailman/listinfo/vfio-users
>
>
>
> _______________________________________________
> vfio-users mailing listvfio-users at redhat.comhttps://www.redhat.com/mailman/listinfo/vfio-users
>
>
> _______________________________________________
> vfio-users mailing list
> vfio-users at redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20190814/a7f004c9/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lstopo_429.png
Type: image/png
Size: 40567 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20190814/a7f004c9/attachment.png>