[linux-lvm] potential locking issues

[Jaco Kroon]

Hi All,

I've applied the attached patch to my installation, and the lock files
are no longer unlinked.  Since then a number of problems have gone away:

1.  dmeventd (in spite of running a watch lvs in a terminal) no longer
ends up with an open
2.  I no longer seem to be getting lockups where I can't execute any lvm
commands until I killall -9 dmeventd.
3.  My snapshots no longer seem to go invalid because dmeventd doesn't
extend them (during testing it still happened but I managed to figure
out it's because my threshold for extending was set at 80%, and my
initial LV was 12M, with 1M writes every 2 seconds to the original
partition, and it would then go from <80% to 100% between checks, I
since updated the threshold to 50% and checking at 15s intervals, and
decreased the writes to every 5s which solved it - in production however
I start with 1G snapshots, threshold at 80 % and increase of 20%).

The only effective change of the patch is that it removes the code that
unlinks the lockfile.  To the best of my knowledge no safe way exists to
unlink the lockfile without introducing lock race conditions, as such,
the empty lock file has to remain on disk.

For your consideration,

Kind Regards,
Jaco

On 20/02/2013 22:26, Jaco Kroon wrote:
> Hi All,
>
> LVM2 uses a locking scheme, relying on flock to maintain lock files for
> volume groups, by default /var/lock/lvm/V_${vgname} - these lock files
> are opened, then flock()ed, and eventually either unlocked and later
> locked again, or potentially just unlink()ed with the lock held.
>
> The unlink() can potentially cause the lock to desync and cause
> problems.  Consider the following scenario with three processes
> (ordering is as is, the numbers are process numbers):
>
> 1.  open()
> 2.  open()
> 1.  flock() <-- succeeds
> 2.  flock() <-- blocks.
> 1.  unlink()
> 1.  close() <-- at this point process 2's flock succeeds.
> 3.  open() <-- note that this ends up being a *different* file.
> 3.  flock() <-- succeeds.
>
> At this point both 2 and 3 thinks they have the lock and that's wrong.
>
> I actually saw an instance today where dmeventd had a file descriptor
> open to a deletect V_vggroup lockfile, so this *does* happen in the
> field.  This also explains various lockups i've seen in the past, which
> I later figured out usually happened when dmeventd was running (So i put
> much effort into ensuring dmeventd never ever started up - which helped
> a lot).
>
> Permitting I'm right the fix would be to fix _undo_flock in
> lib/locking/file_locking.c to not unlink the lockfile - ever.  Or any
> other file that is used for locking purposes anywhere in the codebase
> for that matter.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: LVM2.2.02.98-no-flock-unlink.patch
Type: text/x-patch
Size: 522 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/linux-lvm/attachments/20130302/f956cf79/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: jaco.vcf
Type: text/x-vcard
Size: 244 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/linux-lvm/attachments/20130302/f956cf79/attachment.vcf>