[Linux-cluster] GFS2 vs EXT3 performance issue

Jeetendra jeetendra.p at directi.com
Tue Mar 10 05:46:23 UTC 2009

Hi all,

I am doing some postmark tests with gfs2 on a 2 node cluster using a iSCSI
partition on a SAN device configured in multipath
Im using 2 path in multipath and my multipath config is
My multipath is using 2 Intel NIC (one onboard eth1 and one external eth2)
which are conected to decdicated managed switch(HP)
Im using iscsi-initiator-utils- and my iscsi Node conf. is
# iscsiadm -m node,1 iqn.1994-04.jp.co.xxxxxx:rsd.d7x.t.10126.0a000,1 iqn.1994-04.jp.co.xxxxxx:rsd.d7x.t.10126.0b000
The primary NIC (eth0) is used for cluster setup and fencing which is on
different nerwork other than multipath interface

My kernel is on a 64 bit system with cluster-2.03.10 and
openais-0.80.3 installed from source on a centos 5.2 final box
I have created GFS patition on the Iscsi disk using mkfs -t gfs2 -p
lock_nolock -t alpha:gfs -j 4 /dev/mapper/disk1
And My cluster.conf is at http://pastebin.com/m209d6124
My postmark config is at http://pastebin.com/m76ba067  which creates and
reads Huge no of files
GFS2 is mounted with options "/gfsmount type gfs2
Actually there are 25 postmark procs each writing and reading to their own
locatioons in the same gfs2 mount from a single machine

Im using lock_nolock but i have a feeling that locking is still taking place

A lockdump shows zillions of entries like -  H: s:SH f:EH e:0 p:4461
[postmark] gfs2_inode_lookup+0x113/0x1f5 [gfs2] and performance is too slow
Lockdump result for above 2 test are similar and be seen at

The Result of GFS with dlm_lock is

The Result of GFS with nolock and demote_secs set to 86400 is

The Result for ext3 is at

I have verified the glock trimming patch is applied to the code
I have read reports saying GFS2 gives 95% of ext3 performance but my setup
doesn't seem to get anywhere closer
Any suggestions/tips on what i might be doing wrong ?


More information about the Linux-cluster mailing list