[Linux-cluster] Slowness of GFS2

[Steven Whitehouse]

Hi,

On Fri, 2011-04-15 at 11:00 -0400, Nicolas Ross wrote:
> Hi !
> 
> We are slowly migrating services to our new cluster... We curently have an 8
> node cluster with a 16 1tb disk enclosure in 7 x raid1 pairs, with 2 global
> spares.
> 
> the first 2 arrays are in one vg, wich is in turn seperated in a 100 gig, a
> 300 and a 1tb lvs, all in gfs2.
> 
> I use the firts lv as a global utility partition where I put my developement
> directories (source trees for apache, php, etc), and other binary utilities.
> This partition is mounted all the time on all nodes (it's in /etc/fstab).
> 
> Now, if I connect to the first node, do some make distclean, ./configure, 
> make and make
> install in on of my source directory, httpd for instance. The make distclean 
> for exemple takes about 30 seconds or so.
> 
> Now, I logout from the first node and I move to the next node where I need 
> to do such update, and go into the same directory and do the same thing. Now 
> it takes forever. I stoped after 3 or 4 minutes. There was no other node 
> useing that directory at that time.
> 
> Here's my fstab entry :
> 
> /dev/VGa/gfs /gfs gfs2 defaults,noatime,noquota  0 0
> 
> What can I do or what parameter can I tune to help improve this kind of 
> performance ?
> 
Did you do a sync on the node that you were moving away from before
starting work on the new node? That should help to speed up the change
of node. I suspect that the issue is that the utilities are working over
a large number of small files, and those will take some time to migrate
to the new node, since it will be accessing each file sequentially.

There have been a few performance improvements recently which may help,
depending on exactly which kernel you are using at the moment,

Steve.