[libvirt] [PATCH 0/3] several cgroups/cpuset fixes

[Henning Schild]

On Mon, 14 Dec 2015 16:27:54 -0500
John Ferlan <jferlan at redhat.com> wrote:

> 
> 
> On 11/13/2015 11:56 AM, Henning Schild wrote:
> > Hi,
> > 
> > i already explained some of the cgroup problems in some detail so i
> > will not do that again.
> > https://www.redhat.com/archives/libvir-list/2015-October/msg00876.html
> > 
> > I managed to solve some of the problems in the current codebase, and
> > am now sharing the patches. But they are really just half of what i 
> > had to change to get libvirt to behave in a system with isolated
> > cpus.
> > 
> > Other changes/hacks i am not sending here because they do not work
> > for the general case:
> > - create machine.slice before starting libvirtd (smaller than root)
> >   ... and hope it wont grow
> > - disabling cpuset.cpus inheritance in libvirtd
> > - allowing only xml with fully specified cputune
> > - set machine cpuset to (vcpupins | emulatorpin)
> > 
> > I am not sure how useful the individual fixes are, i am sending them
> > as concrete examples for the problems i described earlier. And i am
> > hoping that will start a discussion.
> > 
> > Henning
> > 
> > Henning Schild (3):
> >   util: cgroups do not implicitly add task to new machine cgroup
> >   qemu: do not put a task into machine cgroup
> >   qemu cgroups: move new threads to new cgroup after cpuset is set
> > up
> > 
> >  src/lxc/lxc_cgroup.c   |  6 ++++++
> >  src/qemu/qemu_cgroup.c | 23 ++++++++++++++---------
> >  src/util/vircgroup.c   | 22 ----------------------
> >  3 files changed, 20 insertions(+), 31 deletions(-)
> > 
> 
> 
> The updated code looks fine to me - although it didn't directly git am
> -3 to top of tree - I was able to make a few adjustments to get things
> merged...  Since no one has objected to this ordering change - I've
> pushed.

Sorry the patches where still based on v1.2.19. Thanks for the merge
and accepting them!

Wrong operation ordering within libvirt cgroups (like the ones
fixed by the patches) could still push tasks onto dedicated cpus. And
more importantly other cgroups users can still grab the dedicated cpus
as well. The only reliable solution to prevent that seems to be making
use of the "exclusive" feature of cpusets. And that would imply
changing the cgroups layout of libvirt again. Because sets can not be
partially exclusive and libvirt deals with dedicated cpus and shared
ones.
How to deal with these problems is a discussion that i wanted to get
started with this patch-series. It would be nice to receive general
comments on that. How should we proceed here? I could maybe write an
RFC mail describing the problems again and suggesting changes to
libvirt on a conceptual basis.

But until then maybe people responsible for cgroups in libvirt (Paul
and Martin?) can again look at
https://www.redhat.com/archives/libvir-list/2015-October/msg00876.html
There i described how naive use of cgoups can place tasks on cpus that
are supposed to be isolated/dedicated/exclusive. Even if libvirt does
not make these mistakes it should protect itself against docker,
systemd, ...

Henning