[vfio-users] A couple notes on DPC latency

[Josh Gao]

> Playing around with isolcpus=1-3,5-7 and nohz_full=1-3,5-7 on an Intel
> quad-core w/ HT, effectively leaving only core0 + its thread pair for
> the host (wishing I had a hex core or better)
>
> So... it doesn't seem like it makes much sense to split the threads of a
> core between host and guest (ie. isolcpus=1-3, let 5-7 schedule
> normally) since neither the host nor guest schedulers can hope to get
> that right.  But, making the emulator run on the idle thread pair seems
> to significant improve latency, which may actually prove more worthwhile
> than giving it to the VM as a vCPU.

I've been setting up a VM of my own on a hexacore Xeon, and I did some
experimentation with this. I think what you're seeing is a side effect of your
cpu governor. My guess is that when you're pinning the emulator to its own
core, it's getting clocked down to nothing, and when it's sharing cores with
the guest, the emulator is always on a core that's relatively awake. I saw
similar results that you did with the ondemand governor, but with the governor
for the emulator core set to performance, I'm seeing perfectly flat 1ms bars.

-Josh