[libvirt] [PATCH 4/4] Add documentation for configuration lock managers
Daniel P. Berrange
berrange at redhat.com
Fri Jun 24 13:02:25 UTC 2011
Add a page which documents how to configure lock managers,
focusing on use of sanlock with the QEMU/KVM driver
* docs/locking.html.in: Docs about lock managers
* docs/sitemap.html.in: Add lock manager config to
the deployment section
---
docs/locking.html.in | 212 ++++++++++++++++++++++++++++++++++++++++++++++++++
docs/sitemap.html.in | 4 +
2 files changed, 216 insertions(+), 0 deletions(-)
create mode 100644 docs/locking.html.in
diff --git a/docs/locking.html.in b/docs/locking.html.in
new file mode 100644
index 0000000..068857d
--- /dev/null
+++ b/docs/locking.html.in
@@ -0,0 +1,212 @@
+<?xml version="1.0"?>
+<html>
+ <body>
+ <h1>Virtual machine disk locking</h1>
+
+ <ul id="toc"></ul>
+
+ <p>
+ This page describes how to ensure a single disk cannot be
+ used by more than one running VM at a time, across any
+ host in a network. This is critical to avoid data corruption
+ of guest files systems that are not cluster aware.
+ </p>
+
+ <h2><a name="plugins">Lock manager plugins</a></h2>
+
+ <p>
+ libvirt includes a pluggable framework for lock managers,
+ which hypervisor drivers can use to ensure safety for
+ guest domain disks, and potentially other resources.
+ At this time there are only two plugin implementations,
+ a "no op" implementation which does absolutely nothing,
+ and a <a href="https://fedorahosted.org/sanlock/">sanlock</a> implementation which uses
+ the Disk Paxos algorithm to ensure safety.
+ </p>
+
+ <h2><a name="sanlock">Sanlock daemon setup</a></h2>
+
+ <p>
+ On many operating systems, the <strong>sanlock</strong> plugin
+ is distributed in a sub-package which needs to be installed
+ separately from the main libvirt RPM. On a Fedora/RHEL host
+ this can be done with the <code>yum</code> command
+ </p>
+
+ <pre>
+ $ su - root
+ # yum install libvirt-lock-sanlock
+ </pre>
+
+ <p>
+ The next step is to start the sanlock daemon. For maximum
+ safety sanlock prefers to have a connection to a watchdog
+ daemon. This will cause the entire host to be rebooted in
+ the event that sanlock crashes / terminates abnormally.
+ To start the watchdog daemon on a Fedora/RHEL host
+ the following commands can be run:
+ </p>
+
+ <pre>
+ $ su - root
+ # chkconfig wdmd on
+ # service wdmd start
+ </pre>
+
+ <p>
+ Once the watchdog is running, sanlock can be started
+ as follows
+ </p>
+
+ <pre>
+ # chkconfig sanlock on
+ # service sanlock start
+ </pre>
+
+ <p>
+ <em>Note:</em> if you wish to avoid the use of the
+ watchdog, add the following line to <code>/etc/sysconfig/sanlock</code>
+ before starting it
+ </p>
+
+ <pre>
+ SANLOCKOPTS="-w 0"
+ </pre>
+
+ <p>
+ The sanlock daemon must be started on every single host
+ that will be running virtual machines. So repeat these
+ steps as neccessary.
+ </p>
+
+ <h2><a name="sanlockplugin">libvirt sanlock plugin configuration</a></h2>
+
+ <p>
+ Once the sanlock daemon is running, the next step is to
+ configure the libvirt sanlock plugin. There is a separate
+ configuration file for each libvirt driver that is using
+ sanlock. For QEMU, we will edit <code>/etc/libvirt/qemu-sanlock.conf</code>
+ There is one mandatory parameter that needs to be set,
+ the <code>host_id</code>. This is a integer between
+ 1 and 2000, which must be set to a <strong>unique</strong>
+ value on each host running virtual machines
+ </p>
+
+ <pre>
+ $ su - root
+ # augtool -s set /files/etc/libvirt/qemu-sanlock.conf/host_id 1
+ </pre>
+
+ <p>
+ Repeat this on every host, changing <strong>1</strong> to a
+ unique value for the host.
+ </p>
+
+ <h2><a name="sanlockstorage">libvirt sanlock storage configuration</a></h2>
+
+ <p>
+ The sanlock plugin needs to create leases in a directory
+ that is on a filesystem shared between all hosts running
+ virtual machines. Obvious choices for this include NFS,
+ or GFS2. The libvirt sanlock plugin expects its lease
+ directory be at <code>/var/lib/libvirt/sanlock</code>
+ so update the host's <code>/etc/fstab</code> to mount
+ a suitable shared/cluster filesystem at that location
+ </p>
+
+ <pre>
+ $ su - root
+ # echo "some.nfs.server:/export/sanlock /var/lib/libvirt/sanlock nfs hard,nointr 0 0" >> /etc/fstab
+ # mount /var/lib/libvirt/sanlock
+ </pre>
+
+ <p>
+ In terms of storage requirements, if the filesystem
+ uses 512 byte sectors, you need to allow for <code>1MB</code>
+ of storage for each guest disk. So if you have a network
+ with 20 virtualization hosts, each running 50 virtual
+ machines and an average of 2 disks per guest, you will
+ need <code>20*50*2 == 2000 MB</code> of storage for
+ sanlock.
+ </p>
+
+
+ <p>
+ On one of the hosts on the network is it wise to setup
+ a cron job which runs the <code>virt-sanlock-cleanup</code>
+ script periodically. This scripts deletes any lease
+ files which are not currently in use by running virtual
+ machines, freeing up disk space on the shared filesystem.
+ Unless VM disks are very frequently created + deleted
+ it should be sufficient to run the cleanup once a week.
+ </p>
+
+ <h2><a name="qemuconfig">QEMU/KVM driver configuration</a></h2>
+
+ <p>
+ The QEMU/KVM driver is fully integrated with the lock
+ manager framework as of release <span>0.9.3</span>.
+ The out of the box configuration, however, currently
+ uses the <strong>nop</strong> lock manager plugin.
+ To get protection for disks, it is thus neccessary
+ to reconfigure QEMU to activate the <strong>sanlock</strong>
+ driver. This is achieved by editing the QEMU driver
+ configuration file (<code>/etc/libvirt/qemu.conf</code>)
+ and changing the <code>lock_manager</code> configuration
+ tunable.
+ </p>
+
+ <pre>
+ $ su - root
+ # augtool -s set /files/etc/libvirt/qemu.conf/lock_manager sanlock
+ # service libvirtd restart
+ </pre>
+
+ <p>
+ If all went well, libvirtd will have talked to sanlock
+ and created the basic lockspace. This can be checked
+ by looking for existance of the following file
+ </p>
+
+ <pre>
+ # ls /var/lib/libvirt/sanlock/
+ __LIBVIRT__DISKS__
+ </pre>
+
+ <p>
+ Every time you start a guest, additional lease files will appear
+ in this directory, one for each virtual disk. The lease
+ files are named based on the MD5 checksum of the fully qualified
+ path of the virtual disk backing file. So if the guest is given
+ a disk backed by <code>/var/lib/libvirt/images/demo.img</code>
+ expect to see a lease <code>/var/lib/libvirt/sanlock/bfa0240911bc17753e0b473688822159</code>
+ </p>
+
+ <p>
+ It should be obvious that for locking to work correctly, every
+ host running virtual machines should have storage configured
+ in the same way. The easiest way todo this is to use the libvirt
+ storage pool capability to configure any NFS volumes, iSCSI targets,
+ or SCSI HBAs used for guest storage. Simply replicate the same
+ storage pool XML across every host. It is important that any
+ storage pools exposing block devices are configured to create
+ volume paths under <code>/dev/disks/by-path</code> to ensure
+ stable paths across hosts. An example iSCSI configuration
+ which ensures this is:
+ </p>
+
+ <pre>
+<pool type='iscsi'>
+ <name>myiscsipool</name>
+ <source>
+ <host name='192.168.254.8'/>
+ <device path='your-iscsi-target-iqn'/>
+ </source>
+ <target>
+ <path>/dev/disk/by-path</path>
+ </target>
+</pool>
+ </pre>
+
+ </body>
+</html>
diff --git a/docs/sitemap.html.in b/docs/sitemap.html.in
index db2963e..897ee94 100644
--- a/docs/sitemap.html.in
+++ b/docs/sitemap.html.in
@@ -73,6 +73,10 @@
<span>Firewall and network filter configuration</span>
</li>
<li>
+ <a href="locking.html">Disk locking</a>
+ <span>Ensuring exclusive guest access to disks</span>
+ </li>
+ <li>
<a href="hooks.html">Hooks</a>
<span>Hooks for system specific management</span>
</li>
--
1.7.4.4
More information about the libvir-list
mailing list