[linux-lvm] disabling udev_sync and udev_rules

[Steven Dake (stdake)]

On 3/16/16, 1:06 AM, "linux-lvm-bounces at redhat.com on behalf of Zdenek
Kabelac" <linux-lvm-bounces at redhat.com on behalf of
zdenek.kabelac at gmail.com> wrote:

>Dne 16.3.2016 v 00:52 Steven Dake (stdake) napsal(a):
>>
>>
>> On 3/15/16, 3:56 PM, "linux-lvm-bounces at redhat.com on behalf of Zdenek
>> Kabelac" <linux-lvm-bounces at redhat.com on behalf of
>> zdenek.kabelac at gmail.com> wrote:
>>
>>> Dne 15.3.2016 v 23:31 Serguei Bezverkhi (sbezverk) napsal(a):
>>>> Hello folks,
>>>>
>>>> While trying to make lvm work within a docker container I came across
>>>> an issue when all lvcreate/lvremove got stuck indefinetly or until
>>>> control-c. When I checked process I noticed lvm was waiting on one
>>>> semaphore, I found that other folks hit similar issue and they fixed
>>>>it
>>>> by setting  udev_sync and udev_rules to 0. It also helped my case too.
>>>>
>>>> I would greatly appreciate if you could share your thought if this
>>>> change in future can potentially have any negative impact.
>>>>
>>>> Thank you
>>>
>>> Hi
>>>
>>>
>>> To 'unblock' stuck processes waiting on udev cookie - you could run:
>>>
>>> 'dmsetup udevcomplete_all'
>>>
>>>
>>> However the key question is - how you could get stuck.
>>> That may need further debugging.
>>>
>>> You would need to expose your OS  version and also version of lvm2 in
>>>use.
>>>
>>> Non working cookies are bad - and disabling udev sync is even more bad
>>> idea...
>>
>> Zdeknek,
>>
>> To expand on what Serguei is doing, he is working on a patch to add
>> LVM2+Iscsi in a container for the Cinder (block storage AAS) project in
>
>
>Hi
>
>Well - this should be the 1st. sentence in the initial email reporting
>the 
>problem.
>
>lvm2 DOES NOT (and CANNOT) work properly inside container.
>
>Devices are not 'containerized' resource.
>This is common bug in 'Docker-land' understanding of Linux kernel.
>That's where the hacks like not using 'udev' sync comes from.

Zdenek,

Just for the sake of the archive, we did manage to get LVM to work inside
a container without modifying the udev sync rules by using --ipc=host to
docker start.  Thanks for the pointers. I hope that other people that run
across this problem can find this thread and use the --ipc=host solution,
since containerized applications are the future and would be bleak without
lvm2 ;)

Regards
-steve


>
>> OpenStack.  He is doing this in the upstream repository here:
>>
>> http://github.com/openstack/klla
>>
>> The LVM processes are running within a container.  I suspect if the
>> process is stuck on a semaphore it has something to do with semaphores
>>not
>> being shared with the host OS, because containers naturally create a
>> contained environment.  There are solutions for things like sockets, but
>> not necessarily for things like semaphores for the container to
>> communicate with the host OS.
>>
>> Is there another mechanism besides semaphores to get lvm2 to communicate
>> with udev?  Turning off udev sync side-steps the problem because then
>>udev
>> is not in the picture.  Some people in our community think this is a
>> security risk, although we assume the servers are completely secure.
>>
>> Your advice welcome on how to solve the problem would be mighty nice :)
>>
>> To see the change in full, check out:
>
>
>The proper way to resolve this is - to have some 'system' service doing
>device for you and then transporting such device to your container.
>Some sort of super-controller daemon.
>
>Device creation is controlled by udev - which runs in your core system.
>It's this udev which is processing kernel event and completes cookie and
>unblocks lvm2 command.
>
>But user really should not confuse what is cgrouped process supposed to
>be 
>doing - it really cannot create device (unlike in full virtual VM) - it
>has 
>wide impact over the whole system - so there must be 'upper-level'
>process 
>controlling this in some way and resolving i.e. name conflicts - sync in
>the 
>system you have just one name space - not per-container namespace - and
>there 
>are more and more troubles ahead...
>
>Anyway - my first advice is to active device as service and pass properly
>created device back to your container via some protocol.
>
>Regards
>
>
>Zdenek
>
>
>_______________________________________________
>linux-lvm mailing list
>linux-lvm at redhat.com
>https://www.redhat.com/mailman/listinfo/linux-lvm
>read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/