[libvirt] image locking?

[Itamar Heim]

The main issue is indeed across nodes. As you pointed out, fnctl() won't
solve the issue for iSCSI.
What about the leased lock mechanism, for example, with libvirt playing
an external watchdog and terminating the qemu process if it cannot renew
the lock, or a central (nfs? Central service) storage to maintain the
global locks.

(would have been very nice if the storage would have played along and
accepted ownership assignment/release, but I guess this is too much to
ask for)

-----Original Message-----
From: libvir-list-bounces at redhat.com
[mailto:libvir-list-bounces at redhat.com] On Behalf Of Daniel P. Berrange
Sent: Wednesday, October 15, 2008 21:59 PM
To: Itamar Heim
Cc: libvir-list at redhat.com
Subject: Re: [libvirt] image locking?

On Wed, Oct 15, 2008 at 10:23:03AM -0700, Itamar Heim wrote:
> Hi,
> 
>  
> 
> I am interested to find out how libvirt envisions image locking.
> 
> i.e., how do we make sure multiple nodes are not trying to access the
> same storage volume, probably causing image corruption.
> 
> I know this can be solved by means of a cluster, but it seems
excessive
> (and not possible in all scenarios).
> 
> Central administration (ovirt-like) is also problematic, since unless
> fencing is assumed, it cannot verify the image is no longer being
used.
> 
> If the image/storage volume could be locked (or leased lock), either
by
> the storage/LVM/NFS preventing from a node access to a specific image,
> or by having libvirt/qemu mutually check the lock before accessing it
> (very simply, in leased lock we can check every X minutes, and kill
the
> process to make sure it honors the lock).

In the domain XML format, the semantics are that every <disk> section
added to a guest config is read-write, with an exclusive lock. To allow
multiple guests to use the same disk, is intended that you add either
<readonly/> or <sharable/> element within the <disk>.

That all said, we only implement this for the Xen driver, handing off
the actuall logic to XenD to perform. That we don't implement this in
the QEMU driver is a clear shortcoming that needs addressing. 

If we only care about locking within the scope of a single host, it is
trivial - libvirtd knows all VMs and their config, so can trivially
ensure the appropriate exlusivity checks are done at time of VM start.

As you point out, ideally this locking would be enforced across hosts
too, in the case of shared storage. Cluster software can't actually
magically solve this for us - it can really only make sure the same 
VM is not started twice. I'm not sure that libivirt can neccessarily
solve it in the general case either, but we can at least make an 
effort in some cases. If, for instance, we were to take a proper
fcntl() lock over the files, this would work for disks backed by a
file on shared filesystems like NFS / GFS. fcntl() locks won't
work on disks backed by iSCSI/SCSI block devices though - and this
will actuall play nicely with clustersoftrware too, since they can
be made to forcably release NFS locks when fencing a node.  It is 
possible that SCSI reservations can help in the FibreChannel case.

So as an immediate option we should perform the exclusivity checks in
libvirt, and also apply fcntl() locks over file backed disks. 

Daniel
-- 
|: Red Hat, Engineering, London   -o-
http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-
http://ovirt.org :|
|: http://autobuild.org       -o-
http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B
9505 :|

--
Libvir-list mailing list
Libvir-list at redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list