[libvirt] simple LXC/libvirt busybox container (Unable to get cgroup)

[Daniel P. Berrange]

On Thu, Dec 10, 2009 at 09:26:39PM +0900, Ryota Ozaki wrote:
> On Thu, Dec 10, 2009 at 9:03 PM, Daniel P. Berrange <berrange at redhat.com> wrote:
> > On Thu, Dec 10, 2009 at 02:22:37AM -0600, Tony Risinger wrote:
> >> i'm trying to get even the simplest busybox container with libvirt+LXC
> >> with very limited success.  I feel l am missing something supremely
> >> simple for me to be hung on this for weeks.
> >>
> >> i dont see anything interesting in domain log, but getting this error
> >> from "LIBVIRT_DEBUG=1 libvirtd":
> >>
> >> 05:27:56.113: error : lxcDomainGetInfo:462 : internal error Unable to
> >> get cgroup for arch-nano
> >> 05:27:56.113: debug : virDomainFree:2004 : domain=0x81d8e68
> >> 05:27:56.113: debug : virUnrefDomain:422 : unref domain 0x81d8e68 arch-nano 1
> >> 05:27:56.113: debug : virReleaseDomain:376 : release domain 0x81d8e68 arch-nano
> >> 05:27:56.113: debug : virReleaseDomain:392 : unref connection 0x81dc0f0 2
> >> 05:27:56.113: debug : remoteSerializeError:141 : prog=536903814 ver=1
> >> proc=16 type=1 serial=4, msg=internal error Unable to get cgroup for
> >> arch-nano
> >>
> >> i've been using this root filesystem layout:
> >>
> >> [root at PHS-001 arch-nano]# tree
> >> .
> >> |-- bin
> >> |   |-- cat -> ../sbin/busybox
> >> |   |-- chdir -> ../sbin/busybox
> >> |   |-- chmod -> ../sbin/busybox
> >> |   |-- ls -> ../sbin/busybox
> >> |   |-- rm -> ../sbin/busybox
> >> |   |-- sh -> ../sbin/busybox
> >> |   `-- vi -> ../sbin/busybox
> >> |-- dev
> >> |   `-- pts
> >> |-- etc
> >> |-- proc
> >> |-- sbin
> >> |   |-- busybox
> >> |   `-- init -> busybox
> >> `-- sys
> >>
> >> all folders besides /bin and /sbin were created by libvirt.  i tried
> >> using the /sbin/init script previously suggested:
> >>
> >> #!/sbin/busybox
> >> sh
> >
> >
> > Sorry, my suggestion was wrong. I forgot that if you have #!/sbin/busybox
> > it will attempt to execute the command matching the name of the script.
> > So it will in fact try to run 'init', rather than 'sh'.
> >
> > Just make the libvirt XML point directly to /bin/sh  instead and it
> > should work. I even tested it this time :-)
> 
> Hem, I still have a problem with ns subsystem enabled. Yes, I can launch
> a container however the cgroup hierarchy is wrong from libvirtd expecting
> like:
> 
> /: libvirtd --daemon
> /5345: /usr/libexec/libvirt_lxc --name
> 
> Daniel, could you confirm how about your cgroup hierarchy?

What you do mean by 'ns' subsystem ?  


# grep cgroup /proc/mounts 
cgroup /dev/cgroups/cpu cgroup rw,relatime,cpuacct,cpu 0 0
cgroup /dev/cgroups/memory cgroup rw,relatime,memory 0 0
cgroup /dev/cgroups/devices cgroup rw,relatime,devices 0 0

# cat /proc/`pgrep libvirtd`/cgroup
32:devices:/sysdefault
16:memory:/sysdefault
12:cpuacct,cpu:/sysdefault

# cat /proc/`pgrep libvirt_lxc`/cgroup
32:devices:/sysdefault/libvirt/lxc/vm1
16:memory:/sysdefault/libvirt/lxc/vm1
12:cpuacct,cpu:/sysdefault/libvirt/lxc/vm1


And the process inside the contanier is PID 12309

# cat /proc/12309/cgroup 
32:devices:/sysdefault/libvirt/lxc/vm1
16:memory:/sysdefault/libvirt/lxc/vm1
12:cpuacct,cpu:/sysdefault/libvirt/lxc/vm1


Which all appears to be correct to me

This is on a Fedora 12 host 2.6.31.6-145.fc12.i686.PAE with

CONFIG_UTS_NS=y
CONFIG_IPC_NS=y
CONFIG_USER_NS=y
CONFIG_PID_NS=y
CONFIG_NET_NS=y
CONFIG_CGROUP_SCHED=y
CONFIG_CGROUPS=y
# CONFIG_CGROUP_DEBUG is not set
CONFIG_CGROUP_NS=y
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_CGROUP_MEM_RES_CTLR=y
CONFIG_CGROUP_MEM_RES_CTLR_SWAP=y
CONFIG_NET_CLS_CGROUP=y


Regards,
Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|