[Linux-cluster] GFS2 vs EXT3 performance issue

[Jeetendra]

Hi all,

I am doing some postmark tests with gfs2 on a 2 node cluster using a iSCSI
partition on a SAN device configured in multipath
Im using 2 path in multipath and my multipath config is
http://pastebin.com/me4facbb
My multipath is using 2 Intel NIC (one onboard eth1 and one external eth2)
which are conected to decdicated managed switch(HP)
Im using iscsi-initiator-utils-6.2.0.870 and my iscsi Node conf. is
# iscsiadm -m node
172.16.1.252:3260,1 iqn.1994-04.jp.co.xxxxxx:rsd.d7x.t.10126.0a000
172.16.2.252:3260,1 iqn.1994-04.jp.co.xxxxxx:rsd.d7x.t.10126.0b000
The primary NIC (eth0) is used for cluster setup and fencing which is on
different nerwork other than multipath interface

My kernel is 2.6.27.9 on a 64 bit system with cluster-2.03.10 and
openais-0.80.3 installed from source on a centos 5.2 final box
I have created GFS patition on the Iscsi disk using mkfs -t gfs2 -p
lock_nolock -t alpha:gfs -j 4 /dev/mapper/disk1
And My cluster.conf is at http://pastebin.com/m209d6124
My postmark config is at http://pastebin.com/m76ba067  which creates and
reads Huge no of files
GFS2 is mounted with options "/gfsmount type gfs2
(rw,noatime,nodiratime,localflocks,localcaching)"
Actually there are 25 postmark procs each writing and reading to their own
locatioons in the same gfs2 mount from a single machine

Im using lock_nolock but i have a feeling that locking is still taking place

A lockdump shows zillions of entries like -  H: s:SH f:EH e:0 p:4461
[postmark] gfs2_inode_lookup+0x113/0x1f5 [gfs2] and performance is too slow
Lockdump result for above 2 test are similar and be seen at
http://203.199.107.58/lockdump.txt

The Result of GFS with dlm_lock is
http://203.199.107.58/Iteration-1-Test2-gfs2-lock_dlm.html

The Result of GFS with nolock and demote_secs set to 86400 is
http://203.199.107.58/Iteration-1-Test2-gfs2-lock_nolock.html

The Result for ext3 is at
http://203.199.107.58/Iteration-1-Test2-ext3.html

I have verified the glock trimming patch is applied to the code
I have read reports saying GFS2 gives 95% of ext3 performance but my setup
doesn't seem to get anywhere closer
Any suggestions/tips on what i might be doing wrong ?

Regards
Jeetendra