[Linux-cluster] How does caching work in GFS1?

[Jeff Sturm]

> -----Original Message-----
> From: linux-cluster-bounces at redhat.com
[mailto:linux-cluster-bounces at redhat.com]
> On Behalf Of Peter Schobel
> Sent: Wednesday, August 11, 2010 3:28 PM
> To: linux clustering
> Subject: Re: [Linux-cluster] How does caching work in GFS1?
> 
> Increasing demote_secs did not seem to have an appreciable effect.

We run some hosts with demote_secs=86400, for what it's worth.  They
tend to go through a "cold start" each morning, but are responsive for
the remainder of the day.

> The du command is a simplification of the use case. Our developers run
> scripts which make tags in source code directories which require
> stat'ing the files.

Gotcha.  I don't have many good suggestions for version control, but I
can offer commiseration.  Some systems are worse than others.
(Subversion for instance tends to create lots of little lock files, and
performs very poorly on just about every filesystem we've tried.)

How much RAM do you have?  All filesystems like plenty of cache.

One thing you can do is run "gfs_tool counters <mount-point>" a few
times during your 20GB test, that may give you some insight.  For
example, does the number of locks increase steadily or does it plateau?
Does it abruptly drop following the test?  Does the "glocks reclaimed"
number accumulate rapidly?  When locks are held, stat() operations tend
to be very fast.  When a lock has to be obtained, that's when they are
slow.

(Any cluster engineers out there, feel free to tell me if any of this is
right or wrong--I've had to base my understanding of GFS on a lot of
experimentation and empirical evidence, not on a deep understanding of
the software.)

-Jeff