[Linux-cluster] gfs mounted but not working

Tue Nov 7 03:19:50 UTC 2006

Hi.

Thanks for the tips.

But, if there is a deadlock problem with memory, what wold be a good
solution to get each one of the drives on my 4 nodes look like one big
shared drive using gfs?

I think that I have a wrong idea of what GFS is, can you explain it to me
please? is only a file system ? is a way to share storage with distributed
lock?

----- Original Message ----- 
From: "Benjamin Marzinski" <bmarzins at redhat.com>
To: "linux clustering" <linux-cluster at redhat.com>
Sent: Monday, November 06, 2006 11:01 PM
Subject: Re: [Linux-cluster] gfs mounted but not working

> On Mon, Nov 06, 2006 at 01:38:40AM -0300, romero.cl at gmail.com wrote:
> > Hi.
> >
> > Now i'm trying this and it works! for now...
> >
> > Two nodes: node3 & node4
> > node4 export his /dev/sdb2 with gnbd_export as "node4_sdb2"
> > node3 import node4's /dev/sdb2 with gnbd_import (new
/dev/gnbd/node4_sdb2)
> >
> > on node3:  gfs_mkfs -p lock_dlm -t node3:node3_gfs -j 4
/dev/gnbd/node4_sdb2
> >                  mount -t gfs /dev/gnbd/node4_gfs /users/home
> >
> > on node4: mount -t gfs /dev/sdb2 /users/home
> >
> > and both nodes can read an write ths same files on /users/home!!!
> >
> > Now i'm going for this:
> >
> > 4 nodes on a dedicated 3com 1Gbit ethernet switch:
> >
> > node2 exporting with gnbd_export /dev/sdb2 as "node2_sdb2"
> > node3 exporting with gnbd_export /dev/sdb2 as "node3_sdb2"
> > node4 exporting with gnbd_export /dev/sdb2 as "node4_sdb2"
> >
> > node1 (main) will import all "nodeX_sdb2" and create a logical volume
named
> > "main_lv" including:
> >
> >     /dev/sdb2 (his own)
> >     /dev/gnbd/node2_sdb2
> >     /dev/gnbd/node3_sdb2
> >     /dev/gnbd/node4_sdb2
> >
> > Next I will try to export the new big logical volume with "gnbd_export"
and
> > then do gnbd_import on each node.
> > With that each node will see "main_lv", then mount it on /users/home as
gfs
> > and get a big shared filesystem to work toghether.
> >
> > Is this the correct way to do the work??? possibly a deadlock???
>
> Sorry. This will not work. There are a couple of problems.
>
> 1. A node shouldn't ever gnbd import a device it has exported.  This can
> cause memory deadlock. When memory pressure is high nodes try to write
> their buffers to disk. Once the buffer is written to disk, the node can
drop
> it from memory, reducing memory pressure. When you do this over gnbd, for
every
> buffer that you write out on the client, a new buffer request come into
the gnbd
> server. If you import a device you have exported (even indirectly through
the
> logical volume on node1 in this setup) that new request just comes back to
you.
> This means that you suddenly double your buffers in memory, just when
memory was
> running low.
>
> The solution is to only access the local device directly, but never
through
> gnbd. Oh, just a note, if you are planning on accessing the local device
> directly, you must not use the "-c" option when you are exporting the
device.
> This will eventually lead to corruption. The "-c" option is only for
dedicated
> gnbd servers.
>
> 2. Theoretically, you could just have every node export the devices to
every
> other node, and then build a logical volume on top of all the devices on
each
> node, but you should not do this. It totally destroys the benefit of
having
> a cluster. Since your GFS filesystem would then depend on having access to
the
> block devices of every machine, if ANY machine in your cluster went down,
the
> whole cluster would crash, because a piece of your filesystem would just
> disappear.
>
>
> Without shared storage, your gnbd server will be a single point of
failure.
> The most common way that people set up gnbd is with one dedicated gnbd
server
> machine, that is only used to serve gnbd blocks, so that it is unlikely to
> crash.
>
> > Sorry if my english isn't very good ;)
> >
> > ----- Original Message ----- 
> > From: "Kevin Anderson" <kanderso at redhat.com>
> > To: "linux clustering" <linux-cluster at redhat.com>
> > Sent: Sunday, November 05, 2006 10:12 PM
> > Subject: Re: [Linux-cluster] gfs mounted but not working
> >
> >
> > > On 11/5/06, *romero.cl at gmail.com <mailto:romero.cl at gmail.com>*
> > > <romero.cl at gmail.com <mailto:romero.cl at gmail.com>> wrote:
> > > >
> > > >     Hi.
> > > >
> > > >     I'm trying your method, but still have a problem:
> > > >
> > > >     Note: /dev/db2/ is a local partition on my second SCSI hard
drive
> > > >     (no RAID)
> > > >     runing on HP ProLiant.
> > > >
> > > GFS requires that all storage is equally accessible by all nodes in
the
> > > cluster.  Your other nodes have no path to the storage you set up so
it
> > > is impossible for them to share the data.
> > >
> > > Kevin
> > >
> > > --
> > > Linux-cluster mailing list
> > > Linux-cluster at redhat.com
> > > https://www.redhat.com/mailman/listinfo/linux-cluster
> >
> > --
> > Linux-cluster mailing list
> > Linux-cluster at redhat.com
> > https://www.redhat.com/mailman/listinfo/linux-cluster
>
> --
> Linux-cluster mailing list
> Linux-cluster at redhat.com
> https://www.redhat.com/mailman/listinfo/linux-cluster