[Linux-cluster] Bottleneck hunting... (GFS Performance)

[HAWKER, Dan]

Hi All,

After a few months of testing, I have finally just setup a production GFS
system. Initially I had trouble with one of the nodes failing (this seems to
have abated, at the moment, but am keeping an eye on things), however
performance seems to be well, just a bit rubbish actually.

I have 2 nodes (at the moment, another couple are waiting in the wings),
each connect to an EMC iSCSI device via Qlogic HBAs over a dedicated
redundant pair of Cisco 3750s.

It seemed pretty snappy in testing (same machines and config, have just
migrated users to the tested setup), however now we actually have production
numbers of ppl accessing it it seems IO bound somewhat, and iostat reports a
(I feel) quite high level of iowait (usually hovers at 3-5% but peaks at up
to 28% on occasions).

Have checked the logs on the EMC and Cisco boxes and they report very low
use, so I can't see that this is the bottleneck, so...

Is there any performance *tips*, places to look at, or known gotchas that I
can look at to try and pin down where the bottleneck lies???

The nodes are presently developer boxes, so my users ssh in and compile
software mostly but also using Samba/NFS to access. Samba/NFS is only
running on one node (as I believe neither is cluster-aware without
significant fiddling).

Any pointers???

TIA

Dan
--

Dan Hawker
Linux System Administrator
Astrium

-- 

This email (including any attachments) may contain confidential and/or privileged information or information otherwise protected from disclosure.
If you are not the intended recipient, please notify the sender immediately, do not copy this message or any attachments and do not use it for any purpose or disclose its content to any person, but delete this message and any attachments from your system.
Astrium disclaims any and all liability if this email transmission was virus corrupted, altered or falsified.
---------------------------------------------------------------------
Astrium Limited, Registered in England and Wales No. 2449259
Registered Office: Gunnels Wood Road, Stevenage, Hertfordshire, SG1 2AS, England