[Libguestfs] LVM flakey failures

[Sam Eiderman]

After upgrading to systemd 248, the problem seems to go away.

I tried to reproduce the issue with:

https://download.libguestfs.org/binaries/appliance/appliance-1.40.1.tar.xz

But also failed (uses systemd 239).

There might be some other differences though (like lvmetad doesn't
exist in debian 10 anymore, etc) - but it does seem like a basic
systemd/udev problem of parsing process output.

I wonder if there is anything libguestfs can do to mitigate this bug.
(since the fix is obviously in .c code deep inside libsystemd).
Some kind of fix / or at least a warning for the next users that will
be affected by this bug.

I originally thought that changing the udev rules to use "bash -c
sleep 1 && dmsetup splitnames" might cause the process to not exit too
quickly, thus avoiding the race in systemd but it seems super hacky
(and I didn't test that this works).

But again, first I think it's important to understand the extent of
this bug throughout the distributions.

Sam


On Thu, Apr 8, 2021 at 6:59 PM Sam Eiderman <sameid at google.com> wrote:
>
> Yes, that's what I understood, but for some reason the appliance stayed on v241 no matter what I did so I started questioning the foundations :-).
> I guess it was just cached somewhere since when I rebuilt the docker from scratch it worked.
>
> One more thing, I had to:
>
> echo "/lib/systemd/libsystemd-shared-248.so" >> appliance/hostfiles.in
>
> Since this file name is obviously version dependent, but now it seems that it's using systemd/udev v248 properly.
>
> Now I'm stress testing with it trying to see if I get a reproduction (hopefully not).
>
> I wonder if this bug is linux wide, at least in terms of libguestfs, and was it just not reported so far.
> I could try using the appliances from libguestfs.org to see if I get a reproduction with them as well.
>
> Something tells me this isn't just debian specific.
>
> Sam
>
> On Thu, Apr 8, 2021 at 4:43 PM Richard W.M. Jones <rjones at redhat.com> wrote:
>>
>> On Thu, Apr 08, 2021 at 04:40:32PM +0300, Sam Eiderman wrote:
>> > However, during libguestfs building I still see this line:
>> >
>> > Get:37 http://deb.debian.org/debian buster/main amd64 systemd amd64
>> > 241-7~deb10u7 [3499 kB]
>> >
>> > So it seems that it downloads the systemd package directly from debian and not
>> > using the one installed on the host docker.
>>
>> But it only downloads that to examine the configuration files.
>> Suggest having a look at how supermin works, it's quite neat!
>>
>> https://libguestfs.org/supermin.1.html#SUPERMIN-APPLIANCES
>>
>> Rich.
>>
>> --
>> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
>> Read my programming and virtualization blog: http://rwmj.wordpress.com
>> virt-builder quickly builds VMs from scratch
>> http://libguestfs.org/virt-builder.1.html
>>