[linux-lvm] disabling udev_sync and udev_rules

[Zdenek Kabelac]

Dne 16.3.2016 v 00:52 Steven Dake (stdake) napsal(a):
>
>
> On 3/15/16, 3:56 PM, "linux-lvm-bounces at redhat.com on behalf of Zdenek
> Kabelac" <linux-lvm-bounces at redhat.com on behalf of
> zdenek.kabelac at gmail.com> wrote:
>
>> Dne 15.3.2016 v 23:31 Serguei Bezverkhi (sbezverk) napsal(a):
>>> Hello folks,
>>>
>>> While trying to make lvm work within a docker container I came across
>>> an issue when all lvcreate/lvremove got stuck indefinetly or until
>>> control-c. When I checked process I noticed lvm was waiting on one
>>> semaphore, I found that other folks hit similar issue and they fixed it
>>> by setting  udev_sync and udev_rules to 0. It also helped my case too.
>>>
>>> I would greatly appreciate if you could share your thought if this
>>> change in future can potentially have any negative impact.
>>>
>>> Thank you
>>
>> Hi
>>
>>
>> To 'unblock' stuck processes waiting on udev cookie - you could run:
>>
>> 'dmsetup udevcomplete_all'
>>
>>
>> However the key question is - how you could get stuck.
>> That may need further debugging.
>>
>> You would need to expose your OS  version and also version of lvm2 in use.
>>
>> Non working cookies are bad - and disabling udev sync is even more bad
>> idea...
>
> Zdeknek,
>
> To expand on what Serguei is doing, he is working on a patch to add
> LVM2+Iscsi in a container for the Cinder (block storage AAS) project in


Hi

Well - this should be the 1st. sentence in the initial email reporting the 
problem.

lvm2 DOES NOT (and CANNOT) work properly inside container.

Devices are not 'containerized' resource.
This is common bug in 'Docker-land' understanding of Linux kernel.
That's where the hacks like not using 'udev' sync comes from.

> OpenStack.  He is doing this in the upstream repository here:
>
> http://github.com/openstack/klla
>
> The LVM processes are running within a container.  I suspect if the
> process is stuck on a semaphore it has something to do with semaphores not
> being shared with the host OS, because containers naturally create a
> contained environment.  There are solutions for things like sockets, but
> not necessarily for things like semaphores for the container to
> communicate with the host OS.
>
> Is there another mechanism besides semaphores to get lvm2 to communicate
> with udev?  Turning off udev sync side-steps the problem because then udev
> is not in the picture.  Some people in our community think this is a
> security risk, although we assume the servers are completely secure.
>
> Your advice welcome on how to solve the problem would be mighty nice :)
>
> To see the change in full, check out:


The proper way to resolve this is - to have some 'system' service doing
device for you and then transporting such device to your container.
Some sort of super-controller daemon.

Device creation is controlled by udev - which runs in your core system.
It's this udev which is processing kernel event and completes cookie and 
unblocks lvm2 command.

But user really should not confuse what is cgrouped process supposed to be 
doing - it really cannot create device (unlike in full virtual VM) - it has 
wide impact over the whole system - so there must be 'upper-level' process 
controlling this in some way and resolving i.e. name conflicts - sync in the 
system you have just one name space - not per-container namespace - and there 
are more and more troubles ahead...

Anyway - my first advice is to active device as service and pass properly 
created device back to your container via some protocol.

Regards


Zdenek