[libvirt] cpu affinity, isolcpus and cgroups

Henning Schild henning.schild at siemens.com
Thu Jul 2 14:02:58 UTC 2015


i am currently looking into realtime VMs using libvirt. My first
starting point was reserving a couple of cores using isolcpus and later
tuning the affinity to place my vcpus on the reserved pcpus.

My first observation was that libvirt ignores isolcpus. Affinity masks
of new qemus will default to all cpus and will not be inherited from
libvirtd. A comment in the code suggests that this is done on purpose.

After that i changed the code to use only the available cpus by
default. But taskset was still showing all 'f's on my qemus. Then i
traced my change down to sched_setaffinity assuming that some other
mechanism might have reverted my hack, but it is still in place.

Libvirt is setting up cgroups and now my suspicion is that cgroups and
taskset might not work well together.
> /sys/fs/cgroup/cpu/machine.slice/machine-qemu\x2dvm1.scope/vcpu0#
> cpuacct.usage_percpu
> 247340587 50851635 89631114 23383025 412639264 1241965 55442753 19923
> 14093629 15863859 27403280 1292195745 82031088 53690508 135826421
> 124915000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Looks like the last 16 cores are not used.

But if i use taskset to ask for the affinity mask i get all 32 cpus.

>  taskset -p `cat tasks`
> pid 12905's current affinity mask: ffffffff

I know that is not strictly libvirt but also a kernel question, still
you guys are probably able to point me to what i am missing here.

> Linux 3.18.11+ #4 SMP PREEMPT RT


More information about the libvir-list mailing list