[Linux-cluster] dlm and IO speed problem <er, might wanna get a coffee first ; )>

[christopher barry]

On Tue, 2008-04-08 at 09:37 -0500, Wendy Cheng wrote:
> gordan at bobich.net wrote:
> >
> >
> >> my setup:
> >> 6 rh4.5 nodes, gfs1 v6.1, behind redundant LVS directors. I know it's
> >> not new stuff, but corporate standards dictated the rev of rhat.
> > [...]
> >> I'm noticing huge differences in compile times - or any home file access
> >> really - when doing stuff in the same home directory on the gfs on
> >> different nodes. For instance, the same compile on one node is ~12
> >> minutes - on another it's 18 minutes or more (not running concurrently).
> >> I'm also seeing weird random pauses in writes, like saving a file in vi,
> >> what would normally take less than a second, may take up to 10 seconds.

Anyway, thought I would re-connect to you all and let you know how this
worked out. We ended up scrapping gfs. Not because it's not a great fs,
but because I was using it in a way that was playing to it's weak
points. I had a lot of time and energy invested in it, and it was hard
to let it go. Turns out that connecting to the NetApp filer via nfs is
faster for this workload. I couldn't believe it either, as my bonnie and
dd type tests showed gfs to be faster. But for the use case of large
sets of very small files, and lots of stats going on, gfs simply cannot
compete with NetApp's nfs implementation. GFS is an excellent fs, and it
has it's place in the landscape - but for a development build system,
the NetApp is simply phenomenal.


Thanks all for your assistance in the many months I have sought and
received advice and help here.

Regards,
Christopher Barry