[Linux-cluster] economy filesystem cluster

[E. de Ruiter]

Hi,

I'm planning to build a webserver cluster, and as part of that I'm 
looking for solutions
that allows every node in the cluster to access the same filesystem. The 
most easy
way would be via nfs, but my requirements state that there should be no 
single point
of failure  (ofcourse not completely possible but the cluster should not 
be affected by the
downtime of 1 machine). A san or other some other piece of extra 
hardware is currently
not possible within the current budget.
The system will have a low number of writes (only some uploaded files 
and some generated
templates but the majority of the load will be reads) but a rsync 
solution or something
like that is not feasible since loadbalancing needs the file to be 
directly available on all
nodes.

What I have:
- 1 loadbalancing machine
- 1 database server
- 2 webfrontends
- 1 management server (slave db / backup load balancer etc)
In the future I plan on adding some extra database servers + webfrontends

All machines are very similar and have (dual) xeon processors. The 
requirements are that
all machines have access to the filesystem, and no single machine may 
affect the availability
of (a part of) the filesystem.
Searching the internet resulted in some possible solutions:
- GFS with only gnbd (http://gfs.wikidev.net/GNBD_installation).
  This only exports the specified partitions over the network and
  has (in my mind) no advantages over using plain nfs (it adds no 
redundancy)
- GFS with gnbd in combination with drbd (mentioned a few times on the 
mailing list).
  This looks promising but I couldn't find a definitive answer to the 
questions
  raised here on the mailinglist:
  - drbd 0.7 only allows 1 node to have write-access. Is it possible to 
construct a simple
    failover scenario without serious risks of corruption when drbd has 
"failed-over"
    but gfs has not.
  - drbd 0.8 seems to have support for multi(2)-master configuration, 
but is it stable
    enough for a production environment and can it work together with gfs
- GFS in combination with clvm (network raid?). Mentioned a few times 
here on
  the mailinglist but most posts claim it is not stable enough, and 
documentation
  seems completely missing.
- economy configuration from the GFS Administrator's Guide
  
(http://www.redhat.com/docs/manuals/csgfs/admin-guide/s1-ov-perform.html#S2-OV-ECONOMY)
  The problem with this is:
  - is there a need to have separate gnbd servers? Or can the gnbd 
servers be run on
    the application servers.
  - it is not documented how to configure this, and it is not clear 
whether this configuration
    gives me the redundancy I want.

What I was thinking of is the following:
- One node acts as a gnbd server
- Each node has his own disk
- Each node mounts a gnbd device.
- Each node creates a raid-1 (own disk + gnbd device)
- GFS is run on top of the raid-1

But it is not clear to me if this is feasible since I rely on a single gnbd
server. Maybe I can have 2 gnbd servers where the disks are synced
with drbd (0.8?), but that creates issues with fencing (according to 
some posts here).
And also the raid-1 should read only from it's local disk and only if that
fails it should read from the gnbd device, but I don't know if that is 
possible.
Or maybe clvm (network raid?) would be an option but I couldn't find
any documentation for that.

Can this be done with gfs / clvm / drbd or are there other solutions more
appropriate for this case? (other filesystems I've seen, like 
pvfs2/intermezzo/lustre,
are either not production ready, abandoned or don't have support for
redundancy)

Thanks,

Eric de Ruiter