[libvirt] Libvirt/LXC (lxcControllerMain:380 : EPOLLHUP from fd 7)

Tony Risinger sweetsinsemilla at gmail.com
Mon Dec 7 21:04:46 UTC 2009 

i have made a busybox container mostly from here:
https://lists.linux-foundation.org/pipermail/containers/2008-September/013323.html
please see sections below; i do not think i am doing anything wrong.
is there somewhere i could download a known working config + root?
also, do i not need to manually create critical devices like
zero/random and friends?  i havent seen anything about this in docs.

thanks


there is nothing in the root now except this:


[root at PHS-001 vps]# tree /vps/dom/arch-nano/
/vps/dom/arch-nano/
|-- bin
|   |-- cat -> ../sbin/busybox
|   |-- chdir -> ../sbin/busybox
|   |-- chmod -> ../sbin/busybox
|   |-- ls -> ../sbin/busybox
|   |-- rm -> ../sbin/busybox
|   |-- sh -> ../sbin/busybox
|   `-- vi -> ../sbin/busybox
|-- dev
|   `-- pts
|-- myc.xml
|-- proc
|-- sbin
|   |-- busybox
|   `-- init
`-- sys


and my /sbin/init looks like this:


[root at PHS-001 vps]# cat /vps/dom/arch-nano/sbin/init
#!/sbin/busybox
sh


config looks like:


[root at PHS-001 vps]# cat /vps/dom/arch-nano/myc.xml
<domain type='lxc'>
  <name>myc</name>
  <memory>500000</memory>
  <os>
    <type>exe</type>
    <init>/sbin/init</init>
  </os>
  <devices>
    <filesystem type='mount'>
      <source dir='/vps/dom/arch-nano'/>
      <target dir='/'/>
    </filesystem>
    <interface type='network'>
      <source network='default'/>
    </interface>
    <console type='pty' />
  </devices>
</domain>


same thing when ran:


[root at PHS-001 vps]# virsh -c lxc:/// create /vps/dom/arch-nano/myc.xml
Domain myc created from /vps/dom/arch-nano/myc.xml

[root at PHS-001 vps]# virsh -c lxc:/// console myc
error: Unable to get domain status
error: internal error Unable to get cgroup for myc

[root at PHS-001 vps]# virsh -c lxc:/// list
 Id Name                 State
----------------------------------
22307 myc                  no state


why/how is busyboy trying to spawn an init? or what is happening here....


.........
22307 ?        Ss     0:00 /usr/lib/libvirt-git/libvirt_lxc --name myc
--console 11 --background --veth veth1
22309 ?        Ss     0:00 /sbin/busybox init
22914 ?        Z      0:00 [init] <defunct>
..........


domain log (pid seem to match above so i dont know why it cant find
it... seems like the cgroup folder matches the pid of the
hypervisor/lxc driver?):


18:07:20.866: debug : lxcControllerRun:540 : Setting up private /dev/pts
18:07:20.883: debug : lxcControllerRun:566 : Mouting 'devpts' on
/vps/dom/arch-nano/dev/pts
18:07:20.883: debug : lxcControllerRun:581 : Opening tty on private
/vps/dom/arch-nano/dev/pts/ptmx
18:07:20.883: debug : virCgroupNew:492 : New group /
18:07:20.883: debug : virCgroupDetect:230 : Detected mount/mapping
0:cpu at /cgroup in /22307
18:07:20.883: debug : virCgroupDetect:230 : Detected mount/mapping
1:cpuacct at /cgroup in /22307
18:07:20.883: debug : virCgroupDetect:230 : Detected mount/mapping
2:cpuset at /cgroup in /22307
18:07:20.883: debug : virCgroupDetect:230 : Detected mount/mapping
3:memory at /cgroup in /22307
18:07:20.883: debug : virCgroupDetect:230 : Detected mount/mapping
4:devices at /cgroup in /22307
18:07:20.883: debug : virCgroupDetect:230 : Detected mount/mapping
5:freezer at /cgroup in /22307
18:07:20.883: debug : virCgroupNew:492 : New group /libvirt
18:07:20.883: debug : virCgroupDetect:230 : Detected mount/mapping
0:cpu at /cgroup in /22307
18:07:20.883: debug : virCgroupDetect:230 : Detected mount/mapping
1:cpuacct at /cgroup in /22307
18:07:20.883: debug : virCgroupDetect:230 : Detected mount/mapping
2:cpuset at /cgroup in /22307
18:07:20.883: debug : virCgroupDetect:230 : Detected mount/mapping
3:memory at /cgroup in /22307
18:07:20.883: debug : virCgroupDetect:230 : Detected mount/mapping
4:devices at /cgroup in /22307
18:07:20.883: debug : virCgroupDetect:230 : Detected mount/mapping
5:freezer at /cgroup in /22307
18:07:20.883: debug : virCgroupMakeGroup:450 : Make group /libvirt
18:07:20.883: debug : virCgroupMakeGroup:462 : Make controller
/cgroup/22307/libvirt/
18:07:20.884: debug : virCgroupCpuSetInherit:416 : Setting up
inheritance / -> /libvirt
18:07:20.884: debug : virCgroupGetValueStr:305 : Get value
/cgroup/22307/cpuset.cpus
18:07:20.884: debug : virCgroupCpuSetInherit:429 : Inherit cpuset.cpus = 0-3
18:07:20.884: debug : virCgroupSetValueStr:275 : Set value
/cgroup/22307/libvirt/cpuset.cpus
18:07:20.884: debug : virCgroupGetValueStr:305 : Get value
/cgroup/22307/cpuset.mems
18:07:20.884: debug : virCgroupCpuSetInherit:429 : Inherit cpuset.mems = 0
18:07:20.884: debug : virCgroupSetValueStr:275 : Set value
/cgroup/22307/libvirt/cpuset.mems
18:07:20.884: debug : virCgroupMakeGroup:462 : Make controller
/cgroup/22307/libvirt/
18:07:20.884: debug : virCgroupMakeGroup:462 : Make controller
/cgroup/22307/libvirt/
18:07:20.884: debug : virCgroupMakeGroup:462 : Make controller
/cgroup/22307/libvirt/
18:07:20.884: debug : virCgroupMakeGroup:462 : Make controller
/cgroup/22307/libvirt/
18:07:20.884: debug : virCgroupMakeGroup:462 : Make controller
/cgroup/22307/libvirt/
18:07:20.884: debug : virCgroupNew:492 : New group /libvirt/lxc
18:07:20.884: debug : virCgroupDetect:230 : Detected mount/mapping
0:cpu at /cgroup in /22307
18:07:20.884: debug : virCgroupDetect:230 : Detected mount/mapping
1:cpuacct at /cgroup in /22307
18:07:20.884: debug : virCgroupDetect:230 : Detected mount/mapping
2:cpuset at /cgroup in /22307
18:07:20.884: debug : virCgroupDetect:230 : Detected mount/mapping
3:memory at /cgroup in /22307
18:07:20.884: debug : virCgroupDetect:230 : Detected mount/mapping
4:devices at /cgroup in /22307
18:07:20.884: debug : virCgroupDetect:230 : Detected mount/mapping
5:freezer at /cgroup in /22307
18:07:20.884: debug : virCgroupMakeGroup:450 : Make group /libvirt/lxc
18:07:20.884: debug : virCgroupMakeGroup:462 : Make controller
/cgroup/22307/libvirt/lxc/
18:07:20.976: debug : lxcContainerStart:832 : clone() returned, 22309
18:07:20.976: debug : virRun:809 : ip link set veth1 netns 22309

On Mon, Dec 7, 2009 at 4:20 AM, Daniel P. Berrange <berrange at redhat.com> wrote:
> On Mon, Dec 07, 2009 at 03:02:53AM -0600, Tony Risinger wrote:
>> hello,
>>
>> i am trying to use libvirt with LXC, and having some issues...
>> everything seems ok when using lxc-* tools.  host box is Arch Linux.
>> ultimately i'm getting this error in the domain log:
>>
>> .....
>> 05:45:31.353: debug : lxcControllerMain:380 : EPOLLHUP from fd 7
>>
>> i was getting this last night in #virt:
>>
>> .....
>> 20:49:57.545: error : lxcFdForward:230 : read of fd 7 failed: Input/output error
>> libvir: Linux Container error : read of fd 7 failed: Input/output error
>
> I believe this essentially says that the container has shutdown.
>
>> -----CONFIGURATION
>>
>> # host ptmx symlinked to pts/ptmx
>> [root at PHS-001 vps]# ls -l /dev/ptmx
>> lrwxrwxrwx 1 root root 8 2009-12-08 04:40 /dev/ptmx -> pts/ptmx
>>
>> # guest ptmx symlinked to pts/ptmx
>> [root at PHS-001 vps]# ls -l /vps/dom/ubuntu/root/dev/ptmx
>> lrwxrwxrwx 1 root root 8 2009-12-06 04:11
>> /vps/dom/ubuntu/root/dev/ptmx -> pts/ptmx
>
> Avoid doing this - libvirt will set that up as part of the container
> bootstrapping processs.
>
>> # guest config...
>> # i have tried the legacy <console type='pty' /> with same results
>> # i dont fully understand this, why do i need to specify a host/source pty?
>> [root at PHS-001 vps]# cat /vps/def/exec/sys/ubuntu.xml
>> <domain type='lxc'>
>>   <name>ubuntu</name>
>>   <memory>500000</memory>
>>   <os>
>>     <type arch='i686'>exe</type>
>>     <init>/sbin/init</init>
>>   </os>
>>   <clock offset='utc'/>
>>   <on_poweroff>destroy</on_poweroff>
>>   <on_reboot>restart</on_reboot>
>>   <on_crash>destroy</on_crash>
>>   <devices>
>>     <filesystem type='mount'>
>>       <source dir='/vps/dom/ubuntu/root'/>
>>       <target dir='/'/>
>>     </filesystem>
>>     <console type='pty'>
>>       <source path='/dev/pts/4'/>
>
> Leave out the <source> here - that's auto-generated when the container
> starts
>
>>       <target port='0'/>
>>     </console>
>>   </devices>
>> </domain>
>>
>> -----PROCEDURE
>>
>> [root at PHS-001 vps]# LIBVIRT_DEBUG=1 libvirtd --daemon
>> 05:42:46.121: debug : virInitialize:278 : register drivers
>> .....
>>
>> [root at PHS-001 vps]# virsh -c lxc:/// define /vps/def/exec/sys/ubuntu.xml
>> Domain ubuntu defined from /vps/def/exec/sys/ubuntu.xml
>>
>> [root at PHS-001 vps]# virsh -c lxc:/// define /vps/def/exec/sys/ubuntu.xml
>> Domain ubuntu defined from /vps/def/exec/sys/ubuntu.xml
>>
>> [root at PHS-001 vps]# virsh -c lxc:/// start ubuntu
>> Domain ubuntu started
>>
>> [root at PHS-001 vps]# virsh -c lxc:/// console ubuntu
>> error: Unable to get domain status
>> error: internal error Unable to get cgroup for ubuntu
>
>
>
>
>>
>> -----LOGS
>>
>> # /var/log/libvirt/lxc/ubuntu.log
>> 05:45:31.126: debug : virCgroupMakeGroup:450 : Make group /libvirt/lxc
>> 05:45:31.126: debug : virCgroupMakeGroup:462 : Make controller
>> /cgroup/1534/libvirt/lxc/
>> 05:45:31.211: debug : lxcContainerStart:832 : clone() returned, 1536
>> 05:45:31.353: debug : lxcControllerMain:380 : EPOLLHUP from fd 7
>
>
>>
>> i've tried endless variations of the <devices> section, but i just
>> cant seem to get anything going with libvirt/LXC once i move the root
>> to a new location for a system container.  same container works fine
>> with lxc-* tools (although i manually make a bridge).  any help is
>> appreciated,
>
> Rather that going straight for a full OS, try just setting up a trivial
> busybox root environment which just takes you intoa busybox shell.
>
> eg, in /sbin/init just have
>
>   #!/sbin/busybox
>
>   sh
>
> (Obviously install busybox in /sbin inside the root env too :-)
>
> Daniel
> --
> |: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
> |: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
> |: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
> |: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
>

More information about the libvir-list mailing list