[linux-lvm] vgmknodes --refresh blocks forever waiting on a semaphore

[Peter Rajnoha]

On 02/19/2015 11:19 PM, Joel Friedly wrote:
> I'm trying to some disk failure testing, and we're using LVM on top of
> raw disks.  After replacing the disk, the LV on the disk is unreadable
> until I run vgmknodes --refresh.  That command hangs forever, but I can
> kill -9 it.  After running the command, everything works again and I can
> read the LV.
> 
> I've seen this twice, so I ran strace the second time and you can find
> the output here:  https://gist.github.com/jfriedly/50fe9134c4bc616f9f90
> and Ctrl-F for "425989"
> 
> On line 4250, LVM sets the semaphore's value to 1, then it immediately
> checks the semaphore's value and confirms that it's 1.
> 
> On line 4253, LVM increments the semaphore's value to 2, then it
> immediately checks the semaphore's value and confirms that it's 2.
> 
> On line 4295, LVM gets the semaphore's value and sees that it's 2, then
> it immediately decrements the value to 1 and then waits indefinitely for
> the value to hit 0.
> 
> 
> Is LVM expecting some other process to decrement the semaphore?

Yes, it's expecting notification from udev rule - the 95-dm-notify.rules
(usually found in /lib/udev/rules.d directory) - that one contains
"dmsetup udevcomplete" call which decrements the semaphore. The --refresh
causes CHANGE udev events to be generated for the LV and LVM waits for
any udev processing to be finished before it continues further.

What Linux distribution is this?

Can you attach (or just send me directly) the output of "vgmknodes --refresh -vvvv"
for more debug info. Also, try running "udevadm monitor --udev --env" just
before executing the vgmknodes command and save the output as well.

Also, please consider filing a bug report for the distribution this
problem is seen in. It's better to track the problem this way as you
can share the debug output and communicate the problem directly with
maintainers of LVM in that distribution - the environment used may
differ in various distributions.

-- 
Peter