[vfio-users] cpu core pinning with multiple cpus
Erik Adler
erik.adler at gmail.com
Sat Sep 12 20:44:11 UTC 2015
Thanks Alex for the feedback. Is this what mean by making the emulator
run on the idle thread pair ?
<vcpu placement='static'>4</vcpu>
<cputune>
<vcpupin vcpu='0' cpuset='2'/>
<vcpupin vcpu='1' cpuset='3'/>
<vcpupin vcpu='2' cpuset='4'/>
<vcpupin vcpu='3' cpuset='5'/>
<emulatorpin cpuset='14-17'/>
</cputune>
.......
<cpu mode='host-passthrough'>
<topology sockets='1' cores='4' threads='1'/>
</cpu>
http://i.imgur.com/GBczQvi.png
Unfortunately the above did not solve the fps issues. Still only have
about 5% of my bare metal speed in that game. Unigine Valley Benchmark
just gave me 93% of bare metal speed.
I am starting to get a feeling there is a problem with Windows
scheduling using host-passthough from my system. Sigh.. then there is
the removed Hyper-V enlightenments to look into.
On Sat, Sep 12, 2015 at 12:23 AM, Erik Adler <erik.adler at gmail.com> wrote:
> Certain games are giving me terrible frame rates on my GTX 970.
> Generally these games are not very demanding when using bare metal.
> Unigine Valley Benchmark is doing fine at about 87% native speeds.
> Same with some other GPU intensive benchmarks. The games that have bad
> fps seem to be taxing the cpu heavily and having latency issues in
> passthough.
>
> I am not sure that I have paired my cores correctly. Using Alex's CPU
> latency script I get the following. Since this is a dual CPU system I
> need to keep everything on the same NUMA node.
> There is a definitive pattern but I am not 100% sure that I see the
> correctional with lstopo.
>
>
> | 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
> --+------------------------------------------------------------------------
> 0| 10 8 7 8 7 8 4 4 4 4 4 4 6 7 7 7 7 7 4 4 4 4 4 4
> 1| 9 10 8 8 8 8 4 4 4 4 4 4 7 7 8 8 8 8 4 4 4 4 4 4
> 2| 8 8 10 8 8 7 4 4 4 4 5 4 7 8 7 8 8 8 4 4 4 4 4 4
> 3| 8 8 8 10 8 8 4 5 4 4 4 4 8 7 8 7 8 8 4 4 4 4 4 4
> 4| 8 8 7 8 10 7 4 4 4 4 4 4 7 8 8 8 7 8 3 4 4 4 4 4
> 5| 8 8 8 8 8 10 4 4 4 4 4 4 7 8 8 7 8 7 4 4 4 4 4 4
> 6| 4 4 4 4 4 4 10 6 6 6 6 6 4 4 3 4 4 4 5 7 6 6 6 6
> 7| 4 4 4 5 4 4 5 10 7 7 7 7 4 4 4 5 5 4 6 7 7 7 7 7
> 8| 4 5 4 5 4 5 6 7 10 7 7 6 4 4 5 4 4 5 5 6 6 8 8 7
> 9| 5 5 5 4 4 4 6 7 8 10 8 8 5 4 5 5 5 4 6 8 7 7 8 8
> 10| 4 4 4 4 4 3 5 6 6 6 10 6 4 4 4 4 4 4 5 6 6 6 5 6
> 11| 3 3 4 3 4 4 5 5 6 6 6 10 3 3 4 4 4 4 5 6 6 6 6 5
> 12| 7 8 8 8 8 7 4 5 4 4 4 4 10 8 8 8 8 8 4 5 4 4 5 5
> 13| 8 7 8 7 7 7 3 4 4 4 4 4 8 10 8 8 7 7 4 4 4 4 4 4
> 14| 8 8 7 8 8 8 4 4 4 4 4 4 7 8 10 8 7 8 4 5 4 4 4 4
> 15| 8 7 8 6 8 8 4 4 4 4 4 4 7 7 8 10 8 7 4 5 4 4 4 4
> 16| 9 8 9 9 8 9 4 5 5 4 5 5 8 8 9 9 10 9 4 5 5 4 5 5
> 17| 8 7 8 8 8 7 4 4 4 4 4 4 7 8 8 8 7 10 4 4 4 4 4 4
> 18| 4 4 4 4 4 4 5 7 6 7 7 7 4 4 4 4 4 4 10 6 7 6 7 7
> 19| 5 5 4 4 4 4 6 6 7 7 7 6 4 4 5 4 4 4 6 10 8 7 7 7
> 20| 4 5 4 4 4 4 6 8 7 8 8 8 4 4 5 3 4 4 6 8 10 8 8 8
> 21| 4 4 4 4 4 4 6 5 6 5 6 6 4 4 4 4 4 4 5 6 6 10 7 6
> 22| 5 4 4 5 4 5 6 7 7 7 7 8 4 5 5 5 5 5 6 8 8 8 10 8
> 23| 4 4 4 4 4 4 6 7 7 6 7 6 4 4 4 4 4 4 5 7 6 7 7 10
>
> https://i.imgur.com/PQvT2oR.png
>
> Looking at lstopo (url) I have hopefully mapped out my hardware
> correctly. In numa node “0” I can see my GTX 970 on PCI 10de:13c2 .
>
> http://i.imgur.com/GBczQvi.png
>
> I am a assuming that if I want to use HT on CPU1 my xml file should
> look like this? Have I done something wrong with how I have pinned
> cores?
>
> <domain type='kvm'>
> <name>Windows</name>
> <uuid>cc52dc82-ce9a-45ff-99e6-a92ab0f42b59</uuid>
> <memory unit='KiB'>16777216</memory>
> <currentMemory unit='KiB'>16777216</currentMemory>
> <vcpu placement='static'>8</vcpu>
> <cputune>
> <vcpupin vcpu='0' cpuset='2'/>
> <vcpupin vcpu='1' cpuset='3'/>
> <vcpupin vcpu='2' cpuset='4'/>
> <vcpupin vcpu='3' cpuset='5'/>
> <vcpupin vcpu='4' cpuset='14'/>
> <vcpupin vcpu='5' cpuset='15'/>
> <vcpupin vcpu='6' cpuset='16'/>
> <vcpupin vcpu='7' cpuset='17'/>
> </cputune>
> <os>
> <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>
> <loader type='rom'>/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
> </os>
> <features>
> <acpi/>
> <apic/>
> <pae/>
> <kvm>
> <hidden state='on'/>
> </kvm>
> <vmport state='off'/>
> </features>
> <cpu mode='host-passthrough'>
> <topology sockets='1' cores='4' threads='2'/>
> </cpu>
> <clock offset='localtime'>
> <timer name='rtc' tickpolicy='catchup'/>
> <timer name='pit' tickpolicy='delay'/>
> <timer name='hpet' present='no'/>
> </clock>
> <on_poweroff>destroy</on_poweroff>
> <on_reboot>restart</on_reboot>
> <on_crash>restart</on_crash>
> <pm>
> <suspend-to-mem enabled='yes'/>
> <suspend-to-disk enabled='yes'/>
> </pm>
> <devices>
> <emulator>/usr/sbin/qemu-system-x86_64</emulator>
> <disk type='file' device='disk'>
> <driver name='qemu' type='raw' cache='none'/>
> <source file='/var/lib/libvirt/images/Windows.raw'/>
> <target dev='vda' bus='virtio'/>
> <boot order='1'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x04'
> function='0x0'/>
> </disk>
> <controller type='usb' index='0' model='ich9-ehci1'>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x7'/>
> </controller>
> <controller type='usb' index='0' model='ich9-uhci1'>
> <master startport='0'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x0' multifunction='on'/>
> </controller>
> <controller type='usb' index='0' model='ich9-uhci2'>
> <master startport='2'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x1'/>
> </controller>
> <controller type='usb' index='0' model='ich9-uhci3'>
> <master startport='4'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x06'
> function='0x2'/>
> </controller>
> <controller type='pci' index='0' model='pci-root'/>
> <controller type='ide' index='0'>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
> function='0x1'/>
> </controller>
> <interface type='network'>
> <mac address='52:54:00:07:81:15'/>
> <source network='default'/>
> <model type='virtio'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
> function='0x0'/>
> </interface>
> <hostdev mode='subsystem' type='pci' managed='yes'>
> <source>
> <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
> </source>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> function='0x0'/>
> </hostdev>
> <hostdev mode='subsystem' type='pci' managed='yes'>
> <source>
> <address domain='0x0000' bus='0x03' slot='0x00' function='0x1'/>
> </source>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x05'
> function='0x0'/>
> </hostdev>
> <hostdev mode='subsystem' type='usb' managed='yes'>
> <source>
> <vendor id='0x0518'/>
> <product id='0x0001'/>
> </source>
> </hostdev>
> <hostdev mode='subsystem' type='usb' managed='yes'>
> <source>
> <vendor id='0x046d'/>
> <product id='0xc24a'/>
> </source>
> </hostdev>
> <memballoon model='virtio'>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x08'
> function='0x0'/>
> </memballoon>
> </devices>
> </domain>
>
> I would like to add that I have changed the Windows 10 guest register
> to use MSI for the GPU. That cleared up the audio issues I had when
> the GPU was in pw.
>
> Any advice on my xml would be much appreciated.
>
> Is there anyway to "debug" if this is a result of the removed Hyper-V
> enlightenments such as ;
>
> <timer name='hypervclock' present='yes'/>
>
> All the best
> onryo
>
> PS for formatting I am asking here too
> https://virtualkvm.com/viewtopic.php?pid=1000#p1000
More information about the vfio-users
mailing list