[Linux-cluster] Re: Why GFS is so slow? What it is waiting for?

[Klaus Steinberger]

> I would appreciate it greatly if you could expand on this.

Yep, I used -r 2048 for my new 6 TByte filesystem. Here some information about 
it:

The default for resource group size:

       -r MegaBytes
              gfs_mkfs will try to make Resource Groups about this big.   The
              default is 256 MB.

From the cluster FAQ:

How can I performance-tune GFS or make it any faster?

You shouldn't expect GFS to perform as fast as non-clustered file systems 
because it needs to do inter-node locking and file system coordination. That 
said, there are some things you can do to improve GFS performance.

    *
      Use -r 2048 on gfs_mkfs and mkfs.gfs2 for large file systems.
      The issue has to do with the size of the GFS resource groups, which is 
an internal GFS structure for managing the file system data. This is an 
internal GFS structure, not to be confused with rgmanager's Resource Groups. 
Some file system slowdown can be blamed on having a large number of RGs. The 
bigger your file system, the more RGs you need. By default, gfs_mkfs carves 
your file system into 256MB RGs, but it allows you to specify a preferred RG 
size. The default, 256MB, is good for average size file systems, but you can 
increase performance on a bigger file system by using a bigger RG size. For 
example, my 40TB file system needs 156438 RGs of 256MB each and whenever GFS 
has to run that linked list, it takes a long time. The same 40TB file system 
can be created with bigger RGs--2048MB--requiring only 19555 of them. The 
time savings is dramatic: It took nearly 23 minutes for my system to read in 
all 156438 RG Structures with 256MB RGs, but only 4 minutes to read in the 
19555 RG Structures for my 2048MB RGs. The time to do an operation like df on 
an empty file system dropped from 24 seconds with 256MB RGs, to under a 
second with 2048MB RGs. I'm sure that increasing the size of the RGs would 
help gfs_fsck's performance as well. Future versions of gfs_mkfs and 
mkfs.gfs2 will dynamically choose an RG size to reduce the RG overhead.

Sincerly,
Klaus

-- 
Klaus Steinberger         Beschleunigerlaboratorium
Phone: (+49 89)289 14287  Am Coulombwall 6, D-85748 Garching, Germany
FAX:   (+49 89)289 14280  EMail: Klaus.Steinberger at Physik.Uni-Muenchen.DE
URL: http://www.physik.uni-muenchen.de/~Klaus.Steinberger/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2002 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/linux-cluster/attachments/20080510/3da9d107/attachment.p7s>