[Linux-cluster] GFS performance.

[Mrugesh Karnik]

On Tuesday 21 Apr 2009 01:01:34 Jeff Sturm wrote:
> > -----Original Message-----
> > From: linux-cluster-bounces at redhat.com
> > [mailto:linux-cluster-bounces at redhat.com] On Behalf Of Ryan Golhar
> > Sent: Monday, April 20, 2009 3:04 PM
> > To: linux clustering
> > Subject: Re: [Linux-cluster] GFS performance.
> >
> > This brings up an interesting question for me....We can 6
> > machines that host a bunch of virtual machines.  I'd like to
> > put the virtual machines on a shared SAN disk.  If one of the
> > physical machines goes down, another one will take over and
> > host a virtual machine.

I've described my setup for the same below.

> Nice.  If your virtualization is based on Xen, you can do live
> migrations over shared storage.  I've done it and it works well.

+1

> > Does it make sense to use GFS to manage the SAN then?
>
> Perhaps I misunderstand, but I don't think there's a need for GFS unless
> you also have shared filesystems.
>
> In our setup each virtual machine has a (dedicated) root filesystem on a
> SAN.  They are all formatted with ext3, because they are not shared.
> (We also have shared non-root volumes that are GFS.)

I use clvm volumes as xvd? devices used with Xen domUs and format them with 
standard ext3. GFS2 is used for sharing the xen configuration files between 
the dom0 nodes so that rgmanager can be used to look after the domUs.

So, basically there are two clusters, the dom0 cluster and the domU cluster. 
In the dom0 cluster, I'm running clvmd, fence_xvmd, gfs2 and rgmanager. The 
domU cluster only runs fence_xvm.

Each dom0 node is an unrestricted, unordered <failoverdomainnode/>. Each 
<vm/> is treated directly as a service under <rm/> in cluster.conf. The domain 
argument to each <vm/> specifies which node to run the domU on, by default. 
Also, be sure to specify migrate="live" to enable Live Migration of the domUs. 
The path argument to <vm/> specifies the path to the domU config files. In my 
setup, they reside on the GFS2 mount point.

The setup works pretty well. The only issue is that when the failed dom0 comes 
back up, the domUs associated with it are automatically migrated back. Its not 
a problem for me as of yet, but I'd still like to know how I can override this 
behaviour, if I want to.

Btw, if you do setup things this way, make sure you use the following in 
/etc/sysconfig/cluster:

RGMGR_OPTS="-N"

This'll prevent rgmanager from stopping the services before starting them.

Mrugesh