[libvirt-users] cgroup error starting domains

[Daniel P. Berrange]

On Fri, Jun 07, 2013 at 01:07:01PM +0200, Matteo Bernardini wrote:
> Thanks Daniel for helping with this :)
> 
> # cat /proc/mounts

[snip]

> cgroup /sys/fs/cgroup cgroup  rw,relatime,bfqio,hugetlb,perf_event,blkio,net_cls,freezer,devices,memory,cpuacct,cpu,cpuset,clone_children

Ok so here you've mounted all the cgroups controllers at the same
place. This is really strongly recommended against. It is better
to mount each controller separately at

   /sys/fs/cgroup/<controller name>

and have /sys/fs/cgroup as a tmpfs.

http://www.freedesktop.org/wiki/Software/systemd/PaxControlGroups

eg to get something along the lines of:

$ ls -l /sys/fs/cgroup/
total 0
drwxr-xr-x. 3 root root  0 Jun  3 09:41 blkio
lrwxrwxrwx. 1 root root 11 Jun  3 09:41 cpu -> cpu,cpuacct
lrwxrwxrwx. 1 root root 11 Jun  3 09:41 cpuacct -> cpu,cpuacct
drwxr-xr-x. 3 root root  0 Jun  3 09:41 cpu,cpuacct
drwxr-xr-x. 3 root root  0 Jun  3 09:41 cpuset
drwxr-xr-x. 3 root root  0 Jun  3 09:41 devices
drwxr-xr-x. 3 root root  0 Jun  3 09:41 freezer
drwxr-xr-x. 3 root root  0 Jun  3 09:41 memory
drwxr-xr-x. 2 root root  0 Jun  3 09:41 net_cls
drwxr-xr-x. 2 root root  0 Jun  3 09:41 perf_event
drwxr-xr-x. 4 root root  0 Jun  3 09:42 systemd
$ grep cgroup /proc/mounts 
tmpfs /sys/fs/cgroup tmpfs rw,seclabel,nosuid,nodev,noexec,mode=755 0 0
cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0
cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpuacct,cpu 0 0
cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0
cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0
cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0
cgroup /sys/fs/cgroup/net_cls cgroup rw,nosuid,nodev,noexec,relatime,net_cls 0 0
cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0
cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0


(obviously the 'system' mount point is not required if not using systemd).

> in /etc/libvirt/qemu.conf I uncommented this line
> 
> cgroup_controllers = [ "cpu", "devices", "memory", "blkio", "cpuset",
> "cpuacct" ]

And this, together with your mount setup is what causes the problem.

Here you have told libvirt to *only* use those 6 listed cgroups controllers,
 but because you've mounted all 9 controllers at the same mount point,
it is impossible for libvirt to satisfy  the config you requested. It can't
use the 6 controllers you asked for, without also using the other 3 you have
excluded in the config.

Either comment out that cgroup_controllers setting (to let libvirt do
auto-detection of mounts), or switch to using a separate mount per
controller, or both.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|