[Linux-cluster] GNBD and CLVM and multipath oh my

[Hansjörg Maurer]

Hi,

I tested a multipath gnbd setup some weeks ago and posted it to the list 
on April 19th.
(relevant part of the thread attached)
It works, but it seems to be very experimental :-)

Greetings

Hansjörg

- SAN (sda1+sdb1)
- 2 Nodes directly attached which  export  sda1+sdb1  via gnbd each
(sda1 and sdb1 form a striped lvm)
- Nodes in the LAN which gnbd-import  sda1+sdb1 from each node
-> noda_sda1 as gnbd0
-> noda_sdb1 as gnbd1
-> nodb_sda1 as gnbd2
-> nodb_sdb1 as gnbd3
- now I created a failover multipath configuration
echo "0 85385412 multipath 0 0 2 1 round-robin 0 1 1 251:0 1000
round-robin 0 1 1 251:2 1000" | dmsetup create dma

echo "0 85385412 multipath 0 0 2 1 round-robin 0 1 1 251:3 1000
round-robin 0 1 1 251:1 1000" | dmsetup create dmb


In this configuration traffic to sda1 goes primaly to noda and traffic
to sdb1 primaly to nodeb.
I adapt lvm.conf not to include /dev/gnbd in the search for volumgroups,
instead /dev/mapper/dm (I get rid of the duplicate volumgroup with this
workaround).
After I start clvmd, I can see the Volume on the client.

With this solution, I have a speedup of about 50% compared to example one
(I think because the striping is done by the client, whereas in example
one the client performs round-robin load-balancing
about differnt pathes and the gnbd server stripes on both disks...)

With
dmsetup  message dma 0  disable_group 1
dmsetup  message dmb 0  disable_group 2
dmsetup  message dma 0  enable_group 1
dmsetup  message dmb 0  enable_group 2
I can switch between the two pathes.

It will be a bit of work is to get the startup scripts work correctly,
because the dmsetup multipath command depends on the major and minor
device ID's of the gnbd-devices of the client, which seem not to bee
persistent,



David Golden wrote:

> On 2005-06-02 17:29:04 -0500, Benjamin Marzinski wrote:
> > First, GNBD does not currently work with multipath. The multipath tools
> > are too SCSI-centric. Sorry.
> >
>
> Oh well.
>
> > either (a) or (b) should work. I would bet that (b) gets you better 
> performance
> > on the worker nodes. gnbd has one server per exported device per client.
>
>
> Well, I'll try (b) so.  The head nodes are near-dedicated to file server
> duty and better performance is good :-)
>
> Thanks for your help!
>
> Best Regards,
>
> David Golden
>
> >
>
> --
> Linux-cluster mailing list
> Linux-cluster at redhat.com
> http://www.redhat.com/mailman/listinfo/linux-cluster
>