[Linux-cluster] nfs/smb slow on gfs2

[Andrew A. Neuschwander]

I have a local only (lock_nolock) gfs2 filesystem which is served to 
clients via smb and nfs v3. Crawling around the filesystem is extremely 
slow over smb and nfs. Stat, ls and find over the network take many 
orders of magnitude longer than I'd expect.

The same tasks are very fast locally on the gfs2 filesystem. Also, for 
comparison, I created an ext3 on the same server and exported it via smb 
and nfs. The same task over nfs or samba on ext3 is nearly as fast as 
doing the task locally.

Is there any way to optimized the interaction between smb/nfs and gfs2?

I'm using gfs2 because I needed a filesystem over 8TB. I'm on CentOS5.3 
(2.6.18-128.1.6.el5).

The storage is locally attached. It is two hardware RAID6 (11 disks each 
+ 1 hotspare) arrays striped together with software raid, then managed 
with lvm2. Both the ext3 and gfs2 are on this storage. Large file 
sequential reads are fast. For example, dd if=file-on-nfs of=/dev/null 
will where file-on-nfs is an 80GB file stored on the gfs2 will max out a 
GigE connection.

Any insight or advice on speeding up these operations would be appreciated.

Thanks,
-Andrew
--
Andrew A. Neuschwander, RHCE
Systems/Software Engineer
College of Forestry and Conservation
The University of Montana
http://www.ntsg.umt.edu
andrew at ntsg.umt.edu - 406.243.6310