[libvirt] Problems with <filesystem type='block'>
Doug Goldstein
cardoe at gentoo.org
Fri Mar 22 04:50:40 UTC 2013
On Sun, Mar 3, 2013 at 11:19 PM, Gao feng <gaofeng at cn.fujitsu.com> wrote:
> Hi Doug,
>
> On 2013/03/04 12:38, Doug Goldstein wrote:
>> On Sun, Mar 3, 2013 at 5:24 PM, Doug Goldstein <cardoe at gentoo.org> wrote:
>>> On Fri, Dec 21, 2012 at 10:15 PM, Lars Kellogg-Stedman <lars at oddbit.com> wrote:
>>>> Using libvirt 1.0.1, I'm trying to start an LXC container using the
>>>> '<filesytem type="block">' syntax, like this:
>>>>
>>>> <filesystem type="block" accessmode="passthrough">
>>>> <source dev="/dev/vg_files/vm-foobar-root" />
>>>> <target dir="/" />
>>>> </filesystem>
>>>>
>>>> The specified block device exists:
>>>>
>>>> # ls -lL /dev/vg_files/vm-foobar-root
>>>> brw-rw---- 1 root disk 253, 19 Dec 21 22:23 /dev/vg_files/vm-foobar-root
>>>>
>>>> If I start the domain, it appears to start without any errors...
>>>>
>>>> # virsh start foobar
>>>> Domain foobar started
>>>>
>>>> ...but it's not actually running. The log files (with loglevel=2)
>>>> don't seem to be very interesting; this is everything from the
>>>> instance log file:
>>>>
>>>> 2012-12-22 04:10:57.862+0000: starting up
>>>> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin LIBVIRT_DEBUG=2 LIBVIRT_LOG_OUTPUTS=2:stderr /usr/lib/libvirt/libvirt_lxc --name foobar --console 22 --security=none --handshake 25 --background --veth veth1
>>>> 2012-12-22 04:10:57.967+0000: 1468: info : libvirt version: 1.0.1
>>>> 2012-12-22 04:10:57.967+0000: 1468: info : lxcCapsInit:151 : No driver, not initializing security driver
>>>> PATH=/bin:/sbin TERM=linux container=lxc-libvirt container_uuid=9041e32e-1df2-00c2-4660-dfa5b41510b7 LIBVIRT_LXC_UUID=9041e32e-1df2-00c2-4660-dfa5b41510b7 LIBVIRT_LXC_NAME=foobar /sbin/init
>>>> 2012-12-22 04:10:58.198+0000: 1: warning : lxcContainerDropCapabilities:1788 : libcap-ng support not compiled in, unable to clear capabilities
>>>> 2012-12-22 04:10:58.198+0000: 1493: warning : lxcControllerClearCapabilities:679 : libcap-ng support not compiled in, unable to clear capabilities
>>>>
>>>> Running an "strace" on the libvirtd process (strace -p <libvirtd_pid>
>>>> -f ...), it doesn't look like libvirt is ever trying to mount the
>>>> referenced filesystem.
>>>>
>>>> Is this supposed to work? It seems like the support for having
>>>> libvirt mount the block device is relatively recent, and I haven't had
>>>> much luck finding examples of other folks using this capability.
>>>>
>>>> Thanks,
>>>>
>>>> -- Lars
>>>
>>> Lars,
>>>
>>> I just gave it a whirl. And I'm able to reproduce the same issue you
>>> are seeing. The issue appears to be that
>>> lxcContainerMountFsBlockAuto() is never being called. It was added in
>>> http://www.redhat.com/archives/libvir-list/2011-August/msg00201.html
>>> Hopefully I'll have more info soon.
>>>
>>> --
>>> Doug Goldstein
>>
>> It appears in the case of filesystem type=block the libvirt_lxc helper
>> just gets wedged right away from first execution.
>>
>> 2013-03-04 04:17:55.443+0000: 7197: debug : virFileMakePathHelper:1272 : path=/v
>> ar/run/libvirt/lxc mode=0777
>> 2013-03-04 04:17:55.443+0000: 7197: debug : virFileClose:72 : Closed fd 18
>> 2013-03-04 04:17:55.443+0000: 7197: debug : virCommandRunAsync:2200 : About to r
>> un PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/u
>> sr/x86_64-pc-linux-gnu/gcc-bin/4.6.3 LIBVIRT_DEBUG=1
>> LIBVIRT_LOG_OUTPUTS=1:stderr /usr/libexec/libvirt_lxc --name testdeb
>> --console 17 --security=none --handshake 20 --background --veth veth1
>> 2013-03-04 04:17:55.444+0000: 7197: debug : virFileClose:72 : Closed fd 21
>> 2013-03-04 04:17:55.444+0000: 7197: debug : virCommandRunAsync:2218 :
>> Command result 0, with PID 7294
>> 2013-03-04 04:17:55.444+0000: 7294: debug : virFileClose:72 : Closed fd 3
>> 2013-03-04 04:17:55.444+0000: 7294: debug : virFileClose:72 : Closed fd 4
>> 2013-03-04 04:17:55.444+0000: 7294: debug : virFileClose:72 : Closed fd 5
>> 2013-03-04 04:17:55.444+0000: 7294: debug : virFileClose:72 : Closed fd 6
>> 2013-03-04 04:17:55.444+0000: 7294: debug : virFileClose:72 : Closed fd 7
>> 2013-03-04 04:17:55.444+0000: 7294: debug : virFileClose:72 : Closed fd 8
>> 2013-03-04 04:17:55.444+0000: 7294: debug : virFileClose:72 : Closed fd 9
>> 2013-03-04 04:17:55.444+0000: 7294: debug : virFileClose:72 : Closed fd 10
>> 2013-03-04 04:17:55.444+0000: 7294: debug : virFileClose:72 : Closed fd 11
>> 2013-03-04 04:17:55.444+0000: 7294: debug : virFileClose:72 : Closed fd 12
>> 2013-03-04 04:17:55.444+0000: 7294: debug : virFileClose:72 : Closed fd 13
>> 2013-03-04 04:17:55.444+0000: 7294: debug : virFileClose:72 : Closed fd 14
>> 2013-03-04 04:17:55.444+0000: 7294: debug : virFileClose:72 : Closed fd 15
>> 2013-03-04 04:17:55.444+0000: 7294: debug : virFileClose:72 : Closed fd 16
>> 2013-03-04 04:17:55.444+0000: 7294: debug : virFileClose:72 : Closed fd 19
>>
>> That's the last time anything with PID 7294 or lxc appears in the log
>> file. I'm unfortunately not getting any lxcContainer debug messages in
>> the log using LIBVIRT_DEBUG=1 LIBVIRT_LOG_OUTPUT=1:stderr
>>
>
>
> Did you check the last version libvirt?
> I can't reproduce this problem with last libvirt.
>
> Thanks!
> Gao
So still trying to figure out what I'm doing wrong with LXC because I
just can't get any joy. So I'll go one issue at a time.
The following VM definition:
<domain type='lxc'>
<name>testdeb</name>
<uuid>df03b2ce-725a-42e2-39e4-d646be8facb3</uuid>
<memory unit='KiB'>332768</memory>
<currentMemory unit='KiB'>332768</currentMemory>
<vcpu placement='static'>1</vcpu>
<os>
<type arch='x86_64'>exe</type>
<init>/sbin/init</init>
</os>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/libexec/libvirt_lxc</emulator>
<filesystem type='block' accessmode='passthrough'>
<source dev='/dev/mapper/vms-testdeb'/>
<target dir='/'/>
</filesystem>
<interface type='network'>
<mac address='52:54:00:95:d5:f2'/>
<source network='default'/>
</interface>
<console type='pty'>
<target type='lxc' port='0'/>
</console>
</devices>
</domain>
Results in /dev/mapper/vms-testdeb being mounted right over my host's
/ partition. Here's the output from mount:
rootfs on / type rootfs (rw)
/dev/sda3 on / type ext4 (rw,noatime,data=ordered)
udev on /dev type devtmpfs
(rw,nosuid,relatime,size=10240k,nr_inodes=497278,mode=755)
devpts on /dev/pts type devpts (rw,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,relatime,mode=755)
/dev/mapper/vms-testdeb on / type ext4 (rw,relatime,data=ordered)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
proc on /proc/sys type proc (ro,relatime)
sysfs on /sys type sysfs (ro,relatime)
tmpfs on /sys/fs/cgroup type tmpfs
(rw,nosuid,nodev,noexec,relatime,size=64k,mode=755)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,relatime,cpuset)
cgroup on /sys/fs/cgroup/cpu type cgroup (rw,relatime,cpu)
cgroup on /sys/fs/cgroup/cpuacct type cgroup (rw,relatime,cpuacct)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,relatime,memory)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,relatime,devices)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,relatime,freezer)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,relatime,blkio)
cgroup on /sys/fs/cgroup/net_prio type cgroup (rw,relatime,net_prio)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc
(rw,nodev,noexec,nosuid)
Now if I do the following:
# mkdir /mnt/testdeb
# mount /dev/mapper/vms-testdeb /mnt/testdeb
And change the definition to:
<filesystem type='mount' accessmode='passthrough'>
<source dir='/mnt/testdeb'/>
<target dir='/'/>
</filesystem>
It at least appears to work. The LXC domain boots but believes it only
has R/O access to its /.
As a side note, I created it with:
# mount /dev/mapper/vms-testdeb /mnt/testdeb
# debootstrap squeeze /mnt/testdeb
--
Doug Goldstein
More information about the libvir-list
mailing list