[vfio-users] RT priorty on vCPU threads destroys everything?

[Michael Slade]

Michael Slade wrote:
> This may not strictly be VFIO-related but it is latency-related. Feel 
> free to suggest another place to post.
>
> I have a VM which has 4 of the host's 8 cores, and has multiple PCI 
> devices passed through - GPU, audio, USB and SATA.
>
> If I give the vCPU threads RT priority (via libvirt XML), and then run 
> `stress -c 4` on the VM, the host would freeze.
>
> If I dedicate the 4 cores to the VM via isolcpus= and CPU pinning, 
> then the host is okay but there are still other issues, e.g. the 
> guest's audio glitches like crazy.
>
> Everything is fine (well mostly fine) if I refrain from setting RT 
> priority on the vCPU threads.
>
> What's happening here? Is this a scheduling bug?
>
> Host and guest are running kernel 5.4.91, and qemu is stable-5.0.

So I noticed after sending this that the sound card's IRQ on the host 
side was on one of the cores that I had reserved via isolcpus=.  I moved 
it off to a non-reserved core and all the audio glitch problems went away.

Is isolcpus not enough to keep host stuff off the specified cores?  Or 
is this a bug?  I'm using kernel 5.9.16.

Also, why would even realtime threads mess with IRQs?  I thought IRQs 
preempted pretty much everything except SMIs and NMIs.