[Linux-cluster] GFS profiling result

[Mark Hlawatschek]

Hi,

during a performance analysis and tuning session, I did some profiling with 
oprofile on GFS and dlm. 
I got some weird results ... 

The installed software is:
RHEL4u5, kernel 2.6.9-55.0.2.ELsmp
GFS:  2.6.9-72.2.0.2
DLM: 2.6.9-46.16.0.1

The configuration includes 2 clusternodes.

I put the following load on one cluster node:

100 processes are doing in parallel: 
- create 1000 files with 100kb size each (ie altogether we have 100.000 files)
- flock 1000 files
- unlink 1000 files.

The following oprofile output shows, that the system spends about 49% 
(75%*65%*) of the time in gfs_unlinked_get.
Looking into the code whe can see, that this is related to unlinked.c:
     53 9394211 58.7081 :                       ul = list_entry(tmp, struct 
gfs_unlinked, ul_list);

It can also be observed, that dlm spends more than 50% of its time in 
searching for hashes...

Is this the expected behaviour or can this be tuned somewhere ?

Thanks,

Mark

oprofile shows the following:
# opreport --long-filenames --threshold 1
  samples|      %|
------------------
168187984 75.4905 /gfs
 37896161 17.0095 /usr/lib/debug/lib/modules/2.6.9-55.0.2.ELsmp/vmlinux
 11686302  5.2453 /dlm

# opreport image:/gfs -l --threshold 1
110838927 65.8899  gfs_unlinked_get
12918468  7.6796  gfs_unlinked_hold
10958430  6.5144  scan_glock
9448317   5.6167  examine_bucket
5504188   3.2720  gfs_unlinked_unlock
3795382   2.2562  trylock_on_glock
3368017   2.0022  unlock_on_glock
1939971   1.1532  run_queue

# opreport image:/dlm -l --threshold 1
samples  %        symbol name
5853674  50.0875  search_hashchain
3726276  31.8842  search_bucket
1506327  12.8890  __find_lock_by_id

-- 
Gruss / Regards,

Dipl.-Ing. Mark Hlawatschek
http://www.atix.de/
http://www.open-sharedroot.org/

**
ATIX Informationstechnologie und Consulting AG
Einsteinstr. 10 
85716 Unterschleissheim
Deutschland/Germany