[Linux-cluster] GFS on 2.6.8.1 more simple performance numbers

[Daniel McNeil]

I had more time to test GFS.  Reminder of the setup
(note: I added more memory so the machines are up to 1GB).
3 machines each:
        2 processor (800 MHZ Pentium 3)
        1GB of memory
        2 100Mb ethernet (1 public, 1 private)
        1 2-port Qlogic FC host adapter
2 F/C sitches cascaded together
1 - 10 disk - dual controller FASTT200 (36GB 10,000rpm drives)

The command run was 'time tar xf /Views/linux-2.6.8.1.tar; 
time sync' where /Views is an NFS mounted file system and 
the current working directory is in a clean file system on
a 5-disk stripe 64k stripe width).  For the 2 node case, 
I ran the command in separate directories on each node.
For comparison, the ext3 file system in on a single scsi
disk in data=ordered.


Tar
---		real		user		sys
ext3 tar	0m6.535s	0m0.429s	0m4.010s
ext3 sync	0m21.953s	0m0.000s	0m0.574s
	
gfs 1 node tar 	1m15.286s 	0m0.787s	0m17.085s
gfs 1 node sync	0m7.734s 	0m0.000s 	0m0.190s

gfs 2 node tar	3m58.337s 	0m0.844s 	0m17.082s
gfs 2 node sync	0m3.119s 	0m0.000s 	0m0.117s
gfs 2 node tar	3m55.147s	0m0.911s	0m17.529s
gfs 2 node sync	0m1.862s	0m0.001s	0m0.043s


du -s linux-2.6.8.1 (after 1st mount)
-----		real		user		sys
ext3 		0m5.361s	0m0.039s	0m0.318s
gfs 1 node	0m46.077s	0m0.097s	0m5.144s
gfs 2 node	0m40.835s	0m0.069s	0m3.218s
gfs 2 node	0m41.313s	0m0.089s	0m3.348s

Doing a 2nd du -s should be cached.  On ext3 is always
seems to be.  On gfs the numbers vary quite a bit.

2nd du -s 
---------
ext3 		0m0.130s	0m0.028s 	0m0.101s
gfs 1 node 	0m20.95s	0m0.075s	0m3.102s
gfs 1 node	0m0.453s 	0m0.044s 	0m0.408s
gfs 2 node	0m0.446s	0m0.046s	0m0.400s
gfs 2 node 	0m0.456s 	0m0.028s	0m0.428s

rm -rf linux-2.6.8.1
--------------------
ext3		0m5.050s 	0m0.019s 	0m0.822s
gfs 1 node	0m28.583s 	0m0.094s 	0m8.354s
gfs 2 node	7m16.295s 	0m0.073s 	0m7.785s
gfs 2 node	8m30.809s	0m0.086s 	0m7.759s


Comment/questions:

Tar on gfs on 1 node is nearly 3x slower than ext3.
Tar on 2 gfs nodes in parallel is showing reverse scaling:
	2 nodes take 4 minutes.

Is there some reason why sync is so fast on gfs?
ext3 shows fast tar then long sync, gfs show long
tar and fairly fast sync.

1 time du is around 8 times slow than ext3.  This must the
time in instantiate and acquire the DLM locks for the
inodes.

Do you know the expected time to get instantiate and acquire a
DLM lock?

rm is 6 times slower on gfs than ext3.  Reverse scaling
on removes happening on 2 nodes in parallel.  These are
in separate directories, so one would not expect DLM
conflicts.

Thoughts?

Daniel