[Linux-cluster] GFS2 vs EXT3+HA-LVM

[Corey Kovacs]

Brem,

Thanks for the response. I do recognize the possible issues with the
tagging, that's why I've not pulled the trigger on this yet. I want to find
out what others have done.

One of the problems I have with GFS1/2 is the time it takes to traverse the
file system. I ran an rsync of a gfs2 file system to an ext3 on the same
machine and it took several hours to even build the initial  list for the
rsync for 2.5 million files. I realize that's a lot of files but when I can
watch the count at approximately 100 p/sec it's a bit ridiculous.

My config is as follows.

5 nodes running RHEL5.4 64 bit
8 GB ram each.
4 Gb FC to an EVA8100 (48 spindles in the disk group)
5 750GB GFS2 file systems with 1GB resource groups.

One suggestion from RedHat was to turn off colors for "ls" and such. I've
also turned off quota's, atime, diratime etc.

Again, I've not pulled the trigger on this because I want to see what others
have done to improve the performance of GFS2. I'd really rather take that
route, but I need better performance than I've been able to get so far

Your comments are appreciated.



Regards


Corey.

On Sat, Sep 26, 2009 at 7:17 PM, brem belguebli <brem.belguebli at gmail.com>wrote:

> Hi,
>
> Before starting arguing, I just want to inform you that this is my own
> personnal opinion on the subject.
>
> This said, I have been using HA-LVM in production with another vendor
> cluster stack (HP not to name it) as I didn't have any other
> alternative.
>
> HA-LVM can be a solution in this case.
>
> Though Redhat recommends the HA-LVM when clusterising in
> Active/passive mode (at least untill RHCS 5.2) with unclustered FS (ie
> ext3) , I  do consider it the weakest option (some security concerns)
> as CLVM  is supposed   to offer a more secured waay of doing things
> (supposed... I'll come to this later).
>
> The main problem I see with HA-LVM (aka hosttags) is that in a
> cluster, any person with root privileges (an unaware Unix admin)
> could, from any cluster node, delete the tags of a service hosted by
> another node and replace them by the local node tags and then activate
> the concerned VG's.
>
> This would lead easily to the possibility of mounting the FS on the
> wrong node though already mounted on the other node.
>
> CLVM (cluster LVM) should not allow this when exclusive activation is
> used. The reality being different, the exclusive activation can be
> bypassed (cf https://bugzilla.redhat.com/show_bug.cgi?id=517900).
>
> Surprising answer from Redhat support about GFS2 performance
>
> Brem
>
>
>
>
>
>
>
> 2009/9/26, Corey Kovacs <corey.kovacs at gmail.com>:
> > I have been trying for a very long time now to put together a cluster
> > (RHEL5.2/3/4 + GFS/GFS2) that would match the performance of our current
> > Tru64 cluster using advfs with limited success. I've tried every tuning
> > technique I can think of and received help from numerous people as well
> as
> > this list. For all that I am thankful.
> >
> > However, I've come to the realization that at this time it's simply not
> > reasonable to expect the same performance from GFS2 as my current setup.
> I
> > have even been told from RedHat support that our use case (general
> purpose
> > file server with applications and home dirs along with some data thrown
> in
> > isn't really a good fit for using GFS2. Not sure I agree with that or
> even
> > understand the motivation behind the comment but I am passed that now.
> The
> > simple truth is for our purposes, I think he may be right and I am now
> > exploring the idea of using ha-lvm with ext3. I'd not set ip up before
> this
> > past week and I have to say it looks promising. Rsyncs, directories with
> > lots of files etc. all behave as I expect them to, fast.
> >
> > That said, how many people are actually using the ha-lvm stuff in
> > production? I don't know anyone using it and have heard little if
> anything
> > on this list about it. I am hoping that's because it 'just works' and
> people
> > leave it at that.
> >
> > Has anyone got any practical experience using this method?  I went
> together
> > fairly straightforward once I got past the lvm tagging, but I'd thought
> I'd
> > ask folks what pitfalls there are if any. Only one I can think of is
> someone
> > not in the know forgetting about the tagging involved and having a tough
> > time managing lvm etc.
> >
> >
> > Any thoughts are appreciated
> >
> >
> > Regards
> >
> >
> > Corey
> >
>
> --
> Linux-cluster mailing list
> Linux-cluster at redhat.com
> https://www.redhat.com/mailman/listinfo/linux-cluster
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/linux-cluster/attachments/20090926/9a3fd186/attachment.htm>