[vfio-users] Brutal DPC Latency - how is yours? check it please and report back

[Bronek Kozicki]

Two things you can improve, IMO
* disable NO_HZ
* use isolcpus to dedicate your pinned CPUs to guest only - this will 
also ensure they are not used for guest  IO.


B.


On 29/02/2016 08:45, Rokas Kupstys wrote:
> Yesterday i figured out my latency problem. All things listed 
> everywhere on internet failed. Last thing i tried was pinning one vcpu 
> to two physical cores and it brought latency down. Now i have FX-8350 
> CPU which has shared FPU for each two cores so maybe thats why. With 
> just this pinning latency now is most of the time just above 1000μs. 
> However under load latency increases. I threw out iothreads and 
> emulator pinning and it did not affect much. Superior latency could be 
> achieved using isolcpus=2-7, however leaving just two cores to host is 
> unacceptable. With that setting latency was around 500μs without load. 
> Good part is that Battlefield3 no longer lags, although i observed 
> increased loading times on textures compared to bare metal. Not so 
> good part is that there still is minor sound skipping/cracking since 
> latency is spiking up under load. That is very disappointing. I also 
> tried performance with two VM cores pinned to 4 host cores - bf3 
> lagged enough to be unplayable. 3 vm cores pinned to 6 host cores was 
> already playable but sound was still cracking. I noticed little 
> difference between that and 4 vm cores pinned to 8 host cores. Be nice 
> if sound could be cleaned up. If anyone have any ideas im all ears. 
> Libvirt xml i use now:
>
>>   <vcpu placement='static'>4</vcpu>
>>   <cputune>
>>     <vcpupin vcpu='0' cpuset='0-1'/>
>>     <vcpupin vcpu='1' cpuset='2-3'/>
>>     <vcpupin vcpu='2' cpuset='4-5'/>
>>     <vcpupin vcpu='3' cpuset='6-7'/>
>>   </cputune>
>>   <features>
>>     <acpi/>
>>     <apic/>
>>     <pae/>
>>     <hap/>
>>     <viridian/>
>>     <hyperv>
>>       <relaxed state='on'/>
>>       <vapic state='on'/>
>>       <spinlocks state='on' retries='8191'/>
>>     </hyperv>
>>     <kvm>
>>       <hidden state='on'/>
>>     </kvm>
>>     <pvspinlock state='on'/>
>>   </features>
>>   <cpu mode='host-passthrough'>
>>     <topology sockets='1' cores='4' threads='1'/>
>>   </cpu>
>>   <clock offset='utc'>
>>     <timer name='rtc' tickpolicy='catchup'/>
>>     <timer name='pit' tickpolicy='delay'/>
>>     <timer name='hpet' present='no'/>
>>     <timer name='hypervclock' present='yes'/>
>>   </clock>
>>
> Kernel configs
>> CONFIG_NO_HZ_FULL=y
>> CONFIG_RCU_NOCB_CPU_ALL=y
>> CONFIG_HZ_1000=y
>> CONFIG_HZ=1000
> I am not convinced 1000 hz tickrate is needed. Default one (300) seems 
> to perform equally as well from looking at latency charts. Did not get 
> chance to test it with bf3 yet however.
>
>
> On 2016.01.12 11:12, thibaut noah wrote:
>

[cut]

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160229/64784928/attachment.htm>