[Linux-cluster] problems with 8 node production gfs cluster

[Nemanja Miletic]

Hello,

I am working for major ISP and we have gfs cluster deployed for our mail
system. Cluster includes six smtp and two pop3 nodes. Nodes are on blade
servers, and are accessing FC HP EVA 5000 storage. Whole cluster is on
the same gigabit subnet. Partition that holds mailboxes is shared. We
are using gentoo linux distribution with 2.6.16-gentoo-r1 kernel and
1.02 linux-cluster.  

We have a problem that in busy hours reading and writing from gfs
partition gets very slow. This causes many processes that need to use
the disk to go in D state. This further causes great load (over 200) on
the machines in the cluster. When we cut the pop3 and smtp access on the
firewall load slowly decreases. At the moment we have a limit on syn
connection on 110 port on our load balancers (LVS based) in order to
control the load.



Thank You,
-- 
Nemanja Miletic, System Engineer
-----
YUnet International  http://www.EUnet.yu
Dubrovacka 35/III, 11000 Belgrade
Tel: +381 11 3305633;  Fax: +381 11 3282760
-----
This e-mail is confidential and intended only for the recipient.
Unauthorized distribution, modification or disclosure of its
contents is prohibited. If you have received this e-mail in error,
please notify the sender by telephone +381 11 3305633.