vm vcpu uses isolated cores

[Martin Kletzander]

On Thu, Sep 02, 2021 at 10:44:06AM +0800, Jiatong Shen wrote:
>Hello,
>
>   I am trying to understand why qemu vm CPU threads uses isolated cpus.
>
>  I have a host which isolates some cpus using isolcpu
>like isolcpus=1,2,3,4,5,7,8,9,10,11. unfortunately, vcpupin does not mask
>out these cpus (vcpupin is still something like ffffffff).

That is because there are use cases which need this.  They isolate cpus
to be used by VM only (sometimes even moving kernel workload from these
cpus) and automatically removing isolcpus from the set would break this
valid behaviour.

>When I log in to the system, seems qemu cpu thread only runs on these
>isolcpus. I do not quite understand this behavior, because I think by using
>isolcpu, kernel schedule will exclude these cpus and thus vcpu thread
>shouldn't use these cores unless taskset explicitly got called.. So my
>question is how does cpu thread got scheduled on isolated cpus?
>

libvirt sets the affinity for VMs because libvirt itself might not be
running on all cores and qemu being a child process would otherwise
inherit the affinity.  We even have this in the documentation and if you
want to limit the cpus it needs to be defined in the XML.

It begs the question whether we should somehow coordinate with the
kernel based on isolcpus, but I am not sure under what circumstances we
should do that and what is the proper way to do that.  I would suggest
you file an issue to discuss this further unless someone comes up with a
clear decision here.

Thank you,
Martin

>Thank you.
>
>-- 
>
>Best Regards,
>
>Jiatong Shen
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/libvirt-users/attachments/20210902/f2af2a32/attachment.sig>