[libvirt] Use flock() instead of fcntl()

[Daniel P. Berrange]

On Thu, Jul 25, 2013 at 12:07:44PM +0200, David Weber wrote:
> Thank you for your quick response!
> 
> Am Donnerstag, 25. Juli 2013, 10:31:40 schrieb Daniel P. Berrange:
> > On Thu, Jul 25, 2013 at 08:23:24AM +0000, David Weber wrote:
> > > Hi,
> > > 
> > > we are interested in using virtlockd on an OCFS2 shared filesystem.
> > > We are now facing the problem that virtlockd uses fcntl() locks which
> > > aren't supported by OCFS2 with the o2cb cluster stack and we want
> > > to avoid using indirect leases.
> > > 
> > > OCFS2 instead supports flock() which is quite similar to fcntl(). I
> > > attached a patch which makes libvirt use flock() *instead* of fcntl()
> > > and it seems to work.
> > > 
> > > NFS on the contrast only supports fcntl() so it should be configurable
> > > which lock type to use.
> > > 
> > > I'm not very experienced with locking, so would such a patch be
> > > acceptable or do you see possible problems with it?
> > 
> > We definitely can't use flock() unconditionally like that, as it is
> > less widely supported than fcntl() locking and is known broken on
> > most network filesystems (flock() will only lock wrt the local node,
> > not network-wide).
> 
> flock() locks are cluster aware in recent versions of OCFS2 and I would try to 
> make it configurable which lock type to use.
>
> > I'm pretty surprised that you can fcntl() is not supported in ocfs2.
> > Are you really sure about that ?
> > 
> > This mail message suggests it is supported
> > 
> >   https://oss.oracle.com/pipermail/ocfs2-users/2009-June/003572.html
> > 
> > "Support for fcntl locking aka file-range locking aka posix locking is
> >  provided by vfs for all file systems. However, that support is appropriate
> >  only for local file systems.
> > 
> >  In ocfs2, we have added support for cluster-aware fcntl locking via
> >  the userspace clustering framework that allows one to use ocfs2 with
> >  different cluster-stacks."
> 
> OCFS2 supports fcntl() only with the userspace cluster stacks (pacemaker and 
> cman) which we want to avoid.

Looking again at flock() I see it cannot support locking of ranges, only
the entire file. This makes it unsuitable as a replacement for libvirt's
use of fcntl() I'm afraid. I can only sugggest you configure OCFS2 so
that it supports fcntl(), or setup virtlockd to use separate indirect
leases on a diffrent shared filesystem, or perhaps try sanlock instead
which doesn't require any special filesystem support.

> SUSE advises to use indirect leases which we also want to avoid:
> https://www.suse.com/documentation//sled11/book_kvm/?page=/documentation//sled11/book_kvm/data/sec_libvirt_storage_locking.html
> 
> "virtlockd's default configuration does not allow you to lock disk files placed 
> on a shared file system (for example NFS or OCFS2). Locking on these file 
> systems requires you to specify a lockspace directory on the VM Host Server by 
> setting"
> 
> Although that's not completely correct because NFS supports fcntl()

That's just badly written explanation for that config setting. It should
really be saying that the default configuration does not provide protection
across multiple hosts for file paths which are not visible via a shared
filesystem. eg a SAN LUN in /dev/sdNNN won't be protected in the default
config.

Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|