[vfio-users] threadripper slowness

[Roger Lawhorn]

$ cat tr3-rad7.sh
#!/bin/bash
#  # of cores to use is passed as a parameter

cd /media/dad/QEMU-SSD/qemu-wd

# use pulseaudio
export QEMU_AUDIO_DRV=pa
export QEMU_AUDIO_TIMER_PERIOD=200
export QEMU_PA_SERVER=/run/user/1000/pulse/native

# display current time and expected boot time for Windows...
boottime1=$(date -d "$starttime today + 0 minutes" +'%I:%M:%S')
boottime2=$(date -d "$starttime today + 1 minutes + 26 seconds" +'%I:%M:%S')
echo "Windows expected to reach desktop at $boottime2..."

#launch windows 7 64bit using SEABIOS (non-uefi bios)....
qemu4.0-system-x86_64 -machine type=q35,accel=kvm -cpu host \
-chardev stdio,id=seabios -device 
isa-debugcon,iobase=0x402,chardev=seabios \
-smp $1,sockets=1,cores=$1 \
-bios /usr/share/seabios/bios.bin \
-m 16G \
-vga none \
-soundhw hda \
-mem-prealloc \
-rtc clock=host,base=localtime \
-device 
ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device 
vfio-pci,host=0c:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=0c:00.1,bus=root.1,addr=00.1,multifunction=on \
-device vfio-pci,host=0d:00.0,bus=root.1,addr=00.2,multifunction=on \
-device virtio-scsi-pci,id=scsi \
-drive 
id=disk0,if=virtio,cache=none,format=raw,file=/media/dad/QEMU-SSD/qemu-wd/windows7.img 
\
-drive 
file=/media/dad/QEMU-SSD/qemu-wd/virtio-win-0.1.126.iso,id=isocd,format=raw,if=none 
-device scsi-cd,drive=isocd \
-netdev type=tap,id=net0,ifname=tap0 \
-device virtio-net-pci,netdev=net0,mac=00:16:3e:00:01:01 \
-usb -device usb-host,vendorid=0x276d,productid=0x1119 \
-boot menu=on \
-boot order=c

exit 0


On 8/11/19 9:59 PM, gamanakis at gmail.com wrote:
> You could probably get rid of all the lag you experience. Possibly qemu 4.0
> provides better topology to the guest. Generally Threadripper CPUs have 2
> NUMA nodes, and you would want to pin the vcpus of the guest to one NUMA
> node, and also allocate RAM from the same NUMA node. You would also have to
> make sure the passthrough devices sit on corresponding PCIe lanes which that
> NUMA node controls. There is lots of good info in reddit.com/r/vfio. Please
> also provide your xml file (or script), and what you use as a virtualized
> hard drive. Is it a passthrough NVMe, or virtio-scsi? In the latter case you
> should also pin iothreads on the same NUMA node as everything else.
> You can see the topology and NUMA nodes of your cpu by running lstopo.
>
> I am eager to see a configuration that uses both NUMA nodes of a
> Threadripper CPU in one guest with minimal latency.
>
>> -----Original Message-----
>> From: vfio-users-bounces at redhat.com <vfio-users-bounces at redhat.com>
>> On Behalf Of Roger Lawhorn
>> Sent: Sunday, August 11, 2019 9:31 PM
>> To: vfio-users at redhat.com
>> Subject: Re: [vfio-users] threadripper slowness
>>
>> Hello,
>>
>> Just wanted to mention that I got rid of a lot of the lag by compiling
> qemu 4.0.
>> I dod not know if its a threadripper issue or not.
>> The lag directly coincides with hard drive access.
>> No hard drive access, no game lag.
>> I have redhat virtio drivers installed for the hard drive and the nic.
>> Are these drivers intel only?
>> They are left only from my i7-4940mx processor install.
>>
>>
>> On 8/9/19 11:51 AM, Roger Lawhorn wrote:
>>> Hello.
>>> I am new to the list.
>>> I have been doing gpu passthrough for almost 5 years now with a i7
>>> 4940mx cpu on a msi gt70 laptop.
>>>
>>> My new PC build is:
>>> MSI x399 carbon pro motherboard : bios xxxxxx.1c0 : svm=enabled:
>>> iommu=enabled
>>> Threadripper 2920 12 core
>>> Radeon pro duo R9 Fury X
>>> Nvidia 980 TI OC
>>>
>>> I was unable to passthrough the nvidia card due to it not getting past
>>> the "Running option rom..." message when debugging.
>>>
>>> The Radeon passed through with flying colors.
>>>
>>> The issue:
>>> I used the radeon to run triple A games in linux using steam for linux
>>> and proton. No problems.
>>>
>>> In Windows 7 in qemu I have passed all 24 cpus to it.
>>> They max out to 80% while playing the same triple A games and the
>>> games lag.
>>> I have never had this issue before.
>>>
>>> Admittedly I am using the same script I used on the i7 to get going.
>>> I had to remove threads from the -smp switch and use cores only.
>>>
>>> I also removed the kvm=off which is used for nvidia cards.
>>>
>>> I am looking for links to articles or direct info on what to do to
>>> optimize the cpus.
>>> When not lagging the radeon performs extremely well.
>>>
>>> I am concerned that perhaps kvm is not being used.
>>>
>>>
>>> Thanks
>>>
>>> _______________________________________________
>>> vfio-users mailing list
>>> vfio-users at redhat.com
>>> https://www.redhat.com/mailman/listinfo/vfio-users
>>>
>> _______________________________________________
>> vfio-users mailing list
>> vfio-users at redhat.com
>> https://www.redhat.com/mailman/listinfo/vfio-users
>