[Linux-cluster] Multicast for GFS?

[Lazar Obradovic]

Just an update, 

I've wrote my own mcast server and client based on the examples i've
got, and they work perfectly with this kind of network setup (separate
vlans and everything else), so I guess problem is actually in cman and
it's mcast interface... 

I'll try to correct this today and send a patch.

On Tue, 2004-08-03 at 19:17, Lazar Obradovic wrote:
> Hi, 
> 
> can someone, please, give some advice about configuring multicast with
> GFS? I know it might go out of topic, but it's perhaps useful for
> others.
> 
> I'd use broadcast instead, but I have a problem that two groups of
> servers sharing the same storage, but that are located in different
> vlans, separated by router-on-a-stick, so I guess I have to use
> multicast.
> 
> I've configured the router for multicast (config is right below), but it
> doesn't seem to work. 
> 
> Here's ascii pic of what I'm trying to make: 
> 
>         +--------+
>         | router |
>         +--------+
>          /       \
> +----------+    +----------+
> | switch A |    | switch B | 
> | vlan 100 |    | vlan 200 | 
> +----------+    +----------+
>      |               |
> +----------+    +----------+
> | server A |    | server B |
> +----------+    +----------+
>      |               |
>   +---------------------+
>   |   san / storage     | 
>   +---------------------+
> 
> and relevant config (that I made this far): 
> 
> router (cisco ios): 
> 
> ip multicast-routing 
> !
> interface FastEthernet0/0
>  description Branch A
>  ip address 1.1.1.1 255.255.255.0 
>  ip pim sparse-dense-mode
>  ip igmp version 1
>  encapsulation dot1q 100
> !
> interface FastEthernet0/1
>  description Branch B
>  ip address 1.1.2.1 255.255.255.0
>  ip pim sparse-dense-mode
>  ip igmp version 1
>  encapsulation dot1q 200
> !
> ip pim send-rp-announce FastEthernet0/0 scope 16
> ip pim send-rp-discovery scope 16
> !
> 
> switch A and switch B are manageable Intel switches (dunno the exact
> model; they are bundled with my IBM Blades), but have IGMP Snooping
> turned on on every interface, and show default cisco pim groups
> (224.0.0.40 and 224.0.0.39) on upstream ports.
> 
> /etc/cluster/cluster.conf for each cluster node is same (only important
> part of config is here, ask for more, if needed): 
> 
> <cluster name="test" config_version="1">
>         <cman>
>                 <multicast addr="224.0.0.11"/>
>         </cman>
>         <nodes>
>                 <node name="node1" votes="1">
>                         <multicast addr="224.0.0.11" interface="eth0"/>
>                 </node>
>                 <node name="node2" votes="1">
>                         <multicast addr="224.0.0.11" interface="eth0"/>
>                 </node>
>         </nodes>
> </cluster>
> 
> hosts ping each other, so networking part, as far as basic ip and
> unicast is concerned, is working properly.
> 
> When starting, cman_tool says: 
> cluster # cman_tool join -d
> multicast address 224.0.0.11
> if eth0 for mcast address 224.0.0.11
> setup up interface for address: node1
> 
> and as I can see from strace, "cman_tool join", this is what happens: 
> 
> socket(0x1f /* PF_??? */, SOCK_DGRAM, 2) = 3
> ioctl(3, 0x780b, 0x2)                   = 0
> setsockopt(3, 0x2 /* SOL_?? */, 109, [6516590], 4) = 0
> [...]
> socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP) = 4
> bind(4, {sa_family=AF_INET, sin_port=htons(6809), sin_addr=inet_addr("224.0.0.11")}, 16) = 0
> socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP) = 5
> bind(5, {sa_family=AF_INET, sin_port=htons(6809), sin_addr=inet_addr("1.1.1.2")}, 16) = 0
> setsockopt(3, 0x2 /* SOL_?? */, 100, "\4\0\0\0\0\0\0\0", 8) = 0
> setsockopt(3, 0x2 /* SOL_?? */, 103, "\5\0\0\0\0\0\0\0", 8) = 0
> setsockopt(3, 0x2 /* SOL_?? */, 101, "\1\344\377\277\3\0\0\0\0\0\0\0\1\0\0\0smtp\0\'\1@\210\0"..., 36) = 0
> close(3)                                = 0
> exit_group(0)                           = ?
> 
> I've checked some programming examples on multicast as well as code for
> cman, and I thing cman_tool/join.c has two problems: 
> 
> - it never seems to issue setsockopt(..., IP_ADD_MEMBERSHIP...), thus,
> never joins the group. I believe the problem is in if (!bcast) check,
> which, if replaced with "if (bhe)" should work fine...
> 
> - it binds the socket with multicast address (fd = 4 in my case) instead
> of local address. If the examples I looked are true, one should bind
> local interface, and then specify mcast address in setsockopt call. 
> 
> Can someone comment this issue? Am I going in completly wrong direction,
> or multicast support isn't ready yet?
-- 
Lazar Obradovic, System Engineer
----- 
laza at YU.net
YUnet International http://www.EUnet.yu
Dubrovacka 35/III, 11000 Belgrade
Tel: +381 11 3119901; Fax: +381 11 3119901
-----
This e-mail is confidential and intended only for the recipient.
Unauthorized distribution, modification or disclosure of its
contents is prohibited. If you have received this e-mail in error,
please notify the sender by telephone +381 11 3119901.
-----