[Linux-cluster] gfs and ext3

[Sergey Skipidarov]

HI,

i have a problem with gfs..
3 nodes: (rhel5.2 x64) connected to san network to eva4400.
and i have one gfs1 file system (about 1TB)shared through three nodes
if i place php-code on gfs fs - i have performance in 10 times lower
than on ext3 fs in the same storage

test i do from one node (1), second and third node doing nothing..
only mounted a gfs partition

when php-code is being on gfs vmstat look like this

procs -----------memory---------- ---swap-- -----io---- --system--
-----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us
sy id wa st
 7  0    308 7488148   3032 176300    0    0   248   152 1080 347400
3 28 68  1  0
 5  0    308 7487032   3032 176312    0    0     1     1 1014 387835
2 31 67  0  0
 1  0    308 7486504   3032 176692    0    0     0     0 1011 386325
2 30 68  0  0
13  0    308 7486380   3032 176700    0    0     0     0 1017 391721
2 31 67  0  0
 5  0    308 7485884   3032 176784    0    0     0     0 1011 378043
3 29 68  0  0
 5  0    308 7485512   3056 176760    0    0     5   321 1030 388592
2 32 66  0  0
 8  0    308 7485388   3056 176864    0    0     0     0 1011 375568
2 30 68  0  0

and on ext3 fs

procs -----------memory---------- ---swap-- -----io---- --system--
-----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
60  0    308 6731144  57744 184440    0    0     0     0 21343 56952
53 42  5  0  0
44  0    308 6730152  57756 184540    0    0     0    60 20975 56834
53 41  6  0  0
48  0    308 6728912  57760 184620    0    0     4     0 21094 60327
52 41  6  0  0
27  0    308 6729284  57764 184792    0    0     1     1 21464 58993
48 36 16  0  0
 8  0    308 6730028  57792 184852    0    0     0  1004 21048 52217
42 29 29  0  0
11  0    308 6730152  57792 184960    0    0     0     0 20631 48064
33 21 46  0  0
 4  0    308 6729656  57804 185040    0    0     0    52 19673 38394
22 13 65  0  0
 4  0    308 6729284  57804 185116    0    0     5     1 18023 32104
19 10 71  0  0
 4  0    308 6729284  57804 185184    0    0     0     0 17620 30895
18 10 73  0  0


context switches on gfs about 380000/s, but on ext3 about 50000/s

a tried tune gfs:
- mount noatime,noquota,nodiratime
- set statfs_fast = 1
- increasing statfs_slots

how i can increase performance ?

--
wbr, Sergey Skipidarov