[Linux-cluster] Is this even possible with GFS, replacing NFS

[Brandon Lamb]

On 1/24/07, Robert Peterson <rpeterso at redhat.com> wrote:
> Brandon Lamb wrote:
> > I am on about day 4 or so of researching filesystems and cluster
> > software, and ready to pull my hair out.
> >
> > First let me describe my situation.
> >
> > I have a 20 disk scsi raid-10. The server that it is connected to has
> > 4 gigs of ram, and exports a single directory via NFS. We only have a
> > 150gig partition, using about 100 of it. We use this for our mail data
> > store, 2.8 million files using maildir format.
> >
> > We have 5 linux boxes connected via NFS over gigabit (NOT using jumbo
> > frames) and they all write to the users maildirs via this nfs mount.
> >
> > Now, this IS working and has been for over 1.5-2 years, however it is
> > slow as hell.
> >
> > Here is where my question comes in. We are planning on splitting the
> > current raid into two 10 disk raid10s so that we can replicate and
> > have a live spare. That part I already am 90% sure of how I will
> > handle. But in rebuilding our mail system I began looking at NFS
> > alternatives and came down to OpenAFS and RedHat GFS.
> >
> > After a ton of reading and a few usenet posts it came down to AFS
> > probably was not the solution and to try GFS.
> >
> > Ok, so my current plan is figuring out how to mount the raid10
> > parition /dev/sda1 on the "server" using iSCSI on the 5 mail heads. I
> > got the iSCSI part working in a test setup with 2 machines, 1 being
> > the sever and 1 being the client. This worked great, doing a test
> > transfer of a 771meg maildir took 24 secs compared to the same test
> > over NFS of 6 mins 32 secs.
> >
> > SO. Now I am at the point that I THINK I should be able to setup iscsi
> > on 5 mail heads to mount this server's /dev/sda1 over gigabit ethernet
> > using jumbo frames on a private switch (so bandwidth should not be a
> > problem?) and then setup GFS as the filesystem for this /dev/sda1 so
> > that all 5 can mount it RW.
> >
> > So is this actually possible or am I wasting my time? At this point I
> > FINALLY got the damn cluster-1.0.3 from cvs to compile with the latest
> > 2.6.19.2 kernel and some hack files from kernel.org for fs/dlm/
> > Kconfig, lowcomms-tcp, locomms-sctp, Makefile and got GFS to compile
> > as kernel modules and the cluster stuff compiled.
> >
> > But I cant seem to get it to work, cluster.conf is complaining about
> > stuff and whatnot, hopefully more reading will give me answers, but is
> > all this work and research in vane?
> >
> > My current test setup is using FC6, kernel 2.6.19.2, i have the
> > cluster-1.0.3, i installed gfs2-utils, i have the iscsi target. From
> > what I can tell I just need to finish getting GFS to work and then i
> > should be able to mount my first gfs filesystem over iscsi on my test
> > client machine.
> >
> > Sorry for the long post, hopefully I didnt forget anything.
> Hi Brandon,
>
> You should be able to accomplish what you've described.
>
> You told us a lot about your environment and what you're trying to do,
> but unfortunately you didn't send us your cluster.conf file or the
> complaint messages you were referring to.
>
> Also, you didn't tell us if the cman service is starting okay on all nodes,
> etc.  You also didn't mention whether you're using clvmd, the clustered
> logical volume manager, etc.
>
> Regards,
>
> Bob Peterson
> Red Hat Cluster Suite
>
> --
> Linux-cluster mailing list
> Linux-cluster at redhat.com
> https://www.redhat.com/mailman/listinfo/linux-cluster

Yes not yet. First I wanted to make sure I could actually do what I
wanted, now I will dive more into actually getting it to work. I'll
post new mails for request for help after I try to get it working on
my own first a little more.

Thanks for replying back! I am a little more inspired to plug away at it now.

=D

Thanks!

Brandon