[libvirt-users] lxc containers won't start in a f24 custom install - odd cgroup fs layout observed

[Thierry Parmentelat]

Hi folks

I use libvirt to programmatically spawn lxc containers
I am facing an issue when migrating from fedora23 to fedora24
I use the stock kernel and libvirt version on both deployments, i.e.:
f23: libvirt-1.2.18.3-2.fc23.x86_64 - kernel 4.5.7-202.fc23.x86_64
f24: libvirt-1.3.3.1-4.fc24.x86_64 - kernel 4.6.3-300.fc24.x86_64


First off, I need to outline that the host installation is done through some ad hoc procedure,
as all this happens in the context of the centrally-managed PlanetLab global infrastructure

I could not reproduce this problem on a host that is installed with the standard fedora install
However I could use any clue that would point at a possible reason for this IMHO odd behaviour from libvirt,
so that I can narrow down the potential flaws in my host installation procedure

At first sight, it looks like the f24 deployment uses a different cgroup fs layout - see below
and I suspect this could be the problem, but I am not aware of anything in my install. procedure that
could cause that change, and as per this document
https://libvirt.org/cgroups.html#systemdScope
this change in the naming scheme looks odd - unless the doc is out of date of course


So again, any clue is most welcome; many thanks in advance -- Thierry

------ 

All is fine with f23/libvirt-1.2.18.3-2.fc23.x86_64
However with f24/libvirt-1.3.3.1-4.fc24.x86_64 my lxc containers won't start
and I am seeing this message in the libvirtd logfile

2016-07-09 11:52:33.416+0000: 3479: debug : virCgroupValidateMachineGroup:317 : Name 'lxc-3555-inrisl1.libvirt-lxc' for controller 'cpu' does not match 'inri_sl1', 'lxc-3555-inrisl1', 'inri_sl1.libvirt-lxc', '\
machine-lxc\x2dinri_sl1.scope' or 'machine-lxc\x2d3555\x2dinrisl1.scope'
2016-07-09 11:52:33.416+0000: 3479: debug : virCgroupNewDetectMachine:1501 : Failed to validate machine name for 'inri_sl1' driver 'lxc'
2016-07-09 11:52:33.416+0000: 3479: error : virLXCProcessStart:1501 : internal error: No valid cgroup for machine inri_sl1

At first it looked like the issue could be linked to the domain name having an underscore (example below  uses inri_sl1 as a domain name)
But trying with a regular name 'plain' without an underscore exhibits a similar issue

-------------------------
Here's the set of files under /sys/fs/cgroup/memory that contains 'inri' in their name
(other susbsytems have similar naming schemes in both cases)

========= f23 / libvirt-1.2.18.3-2.fc23.x86_64  (works fine)

[root at vnode06 log]# cat /etc/fedora-release
Fedora release 23 (Twenty Three)
[root at vnode06 log]# uname -r
4.5.7-202.fc23.x86_64
[root at vnode06 log]# rpm -q libvirt
libvirt-1.2.18.3-2.fc23.x86_64
[root at vnode06 log]# find /sys/fs/cgroup/memory/ -name '*inri*'
/sys/fs/cgroup/memory/machine.slice/machine-lxc\x2dinri_sl1.scope

========= f24 / libvirt-1.3.3.1-4.fc24.x86_64  (container won't start)

[root at vnode05 libvirt]# cat /etc/fedora-release
Fedora release 24 (Twenty Four)
[root at vnode05 libvirt]# uname -r
4.6.3-300.fc24.x86_64
[root at vnode05 libvirt]# rpm -q libvirt
libvirt-1.3.3.1-4.fc24.x86_64
[root at vnode05 libvirt]# find /sys/fs/cgroup/memory/ -name '*inri*'
/sys/fs/cgroup/memory/machine/lxc-3237-inrisl1.libvirt-lxc
/sys/fs/cgroup/memory/machine/lxc-3555-inrisl1.libvirt-lxc
/sys/fs/cgroup/memory/machine/lxc-2989-inrisl1.libvirt-lxc