[Linux-cluster] High availability mail system

[Rainer Duffner]

Robert Olsson wrote:

>Ok, how about performance when using NFS? I´m thinking about the overhead when accesing NFS filesystems. 
>


The trouble is that GFS also has an overhead - especially for Qmail.
In fact, we have what I would call a "long-time evaluation" of qmail + 
GFS running.
While some features (no SPOF) are nice, others (way too many concurrent 
directory-accesses to gain any performance gain in comparison to NFS) 
are not nice at all.
I'm really not a GFS-expert at all, but the way I see it (and was told) 
is that everytime a directory-access occurs, GFS must synchronize this 
to the other cluster-members.
Now, when Qmail delivers a mail, it already takes great care not to 
produce conflicts on (NFS-)shared filesystems, by copying the message 
first to "tmp", then to "new", with timestamp as part of filename etc.
For every file created on the shared directory, though, GFS creates 
locks and lockfiles - which easily doubles or triples the load on the 
SAN. Even though, no two files of the same name are ever created by 
different hosts.
In addition, GFS doesn't seem to have "directory hashing" like FreeBSD 
UFS and others have, as a result access to large directories with many 
files is slow. Running find(1) on the mail-store can bring the cluster 
to halt, so does du(1).

Due to the fact that we also will have to move from cdb-backend to 
mysql, there will be a SPOF anyway and we will be actively evaluating 
going to NFS (more or less back).
But only on a "sane" NFS-platform, most likely Solaris, or FreeBSD - 
don't waste your time with Linux-NFS...


>Do you know about any mailsystem that distribute mail over NFS and do you have any links to performance data?
>  
>

The shared-storage mail-systems only scale to a certain point. The 
numbers are in the 300k-500k ballpark. After that, you have to go 
distributed.
If you have millions of users, go qmail-ldap.

These things are very difficult to benchmark, every site has an 
individual use-profile..




cheers,
Rainer