[libvirt] LVM storage pool with existing mirrored volumes

[Matthias Bolte]

2010/1/17 Daniel P. Berrange <berrange at redhat.com>:
> On Sun, Jan 17, 2010 at 03:44:28PM +0000, Ben Roberts wrote:
>> Hi,
>>
>> I tried to setup a lvm-backed storage pool, reusing an existing volume
>> group which already contains a handful of mirrored logical volumes.
>>
>> For reference, I am using lvm 2.02.56-r3, qemu-kvm 0.12.1.2 and libvirt
>> 0.7.5 on an up-to-date ~amd64 gentoo system.
>>
>> The first issue I encountered is when trying to activate the storage
>> pool, the process fails with "internal error lvs command failed". This
>> appears to be because the output of lvs is slightly different when using
>> mirrored lvs, and is breaking the parser:
>>
>> /sbin/lvs --separator , --noheadings --units b --unbuffered --nosuffix
>> --options lv_name,origin,uuid,devices,seg_size,vg_extent_size bigpool
>>
>> vms,,0PzrRS-8Ljs-SUq0-ywFH-BI3d-RI0V-aibupu,vms_mimage_0(0),vms_mimage_1(0),107374182400,4194304
>>
>> backups,,Snk2NL-oJe3-GpTL-Qihs-ZjjG-xPyj-ufmwCp,backups_mimage_0(0),backups_mimage_1(0),214748364800,4194304
>>
>> The device column of output would normally contain something like
>> "/dev/sda(1234)", but here contains "vms_mimage_0(0),vms_mimage_1(0).
>>
>> This is causing the parser to misinterpret the output, and it believes
>> the lv name to be suffixed with a trailing comma, eg "vms,". When trying
>> to access "/dev/bigpool/vms,", the command errors with File or Directory
>> not Found.
>>
>> When adding the --all parameter to lvs, the hidden mirroring lvs are
>> visible, and these have the expected structure:
>>
>> vms,,0PzrRS-8Ljs-SUq0-ywFH-BI3d-RI0V-aibupu,vms_mimage_0(0),vms_mimage_1(0),107374182400,4194304
>> [vms_mlog],,vnOzg8-ynKc-kM8F-RKdM-xXC0-PJlG-qcgmFh,/dev/sda2(0),4194304,4194304
>> [vms_mimage_0],,2pXjHt-aARa-FOO7-ZL6N-ahpu-ebdr-P6Qxj3,/dev/sdb1(0),107374182400,4194304
>> [vms_mimage_1],,ITd8CT-JpKi-Y2a6-a4jc-2yDa-9VmG-gWgRd8,/dev/sdc1(51200),107374182400,4194304
>>
>> Presumably, without the --all parameter and listing the internal
>> mirroring volumes, libvirt will miscalculate sizes of LVs in the VG.
>
> Yes, the output of mirrored devices is definitely going to cause
> problems for our parsing. This doesn't look all that easy to fix to
> me, but we'll clearly have to figure out some way to cope with this.
>
>
> Daniel
>

We discussed this on IRC. One suggestion was to change the separator
from comma to # for example. This would allow to identify the expected
structure of sections again, because the devices list will still be
separated by a comma. Then the devices section could be parsed in a
second phase.

vms##0PzrRS-8Ljs-SUq0-ywFH-BI3d-RI0V-aibupu#vms_mimage_0(0),vms_mimage_1(0)#107374182400#4194304

But the more complicated question (at least to me) is how to handle
mirrored LVs correctly, because with vms_mimage_* the offset seems to
be always 0, but the underlying LVs may have different offsets also
different from 0, and libvirt uses the offset for size calculations.

Well, I'm not really familiar with LVM, so I can't judge if that's a
real/hard problem. Maybe fixing the parser is sufficient to fix the
problem.

Matthias