[Linux-cluster] Cluster Project FAQ - GFS tuning section]

[Robert Peterson]

Wendell Dingus wrote:
> I don't know where that breaking point is but I believe _we've_ stepped
> over it.
>
> 4-node RHEL3 and GFS6.0 cluster with (2) 2TB filesystems (GULM and no
> LVM) versus
> 3-node RHEL4 (x86_64) and GFS6.1 cluster with (1) 8TB+ filesystem (DLM
> and LVM and way faster hardware/disks)
>
> This is a migration from the former to the latter, so quantity/size of
> files/dirs is mostly identical. Files being transferred from customer
> sites to the old servers never cause more than about 20% CPU load and
> that usually (quickly) falls to 1% or less after the initial xfer
> begins. The new servers run to 100% where they usually remain until the
> transfer completes. The current thinking as far as reason is the same
> thing being discussed here.
>
> I was advised of this mailing list posting by a a RH tech from an open
> ticket I have on this problem.
>
> The older server has 11million and 13million files respectively across
> it's two filesystems. The newer server has about 27million files on it's
> one larger filesystem. It appears I'm going to be forced to blow this
> away and create multiple smaller filesystems (there is a redundant
> backup drive with all the files thank goodness). Unless I hear otherwise
> I'm inclined to limit the new RAID to 2TB filesystems max. So as one
> filesystem fills up faster than the others I suppose we'll forever be
> moving things from one to the other and symlinking it to death. Oh well...
>   
Hi Wendell,

If I understand you correctly, you're having a performance problem here.
If it were me, I guess I'd try some experiments to try to determine 
where the
bottleneck seems to be.  For example, I'd try temporarily switching to Gulm
locking protocol on the new servers to see if that brings the 
performance back.

Wendy Cheng has a performance-related bugzilla open right now in GFS:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=214239

Your problem might/might not be related.  Would you be willing to test a
patch for her?  I don't know if she has a patch ready to try yet.

Regards,

Bob Peterson
Red Hat Cluster Suite