[Linux-cluster] CMAN connectivity problem

[Rick Stevens]

Manuel Bujan wrote:
> Hi people,
>  
> We are trying to set up a two nodes cluster with the latest cvs source 
> from RedHat and when we are going to start both nodes we get that the 
> first node form a quorum without problems, but the second one never get 
> connected at cman level and remain trying to connect with the following 
> error:
>  
> Dec 16 15:30:25 atmail-2 ccsd[3260]: Initial status:: Inquorate
> Dec 16 15:30:28 atmail-2 kernel: CMAN: sending membership request
> Dec 16 15:31:03 atmail-2 last message repeated 21 times
> Our cluster.conf file is the following:
>  
> <?xml version="1.0"?>
> <cluster name="ISQCLUSTER" config_version="2">
>         <cman two_node="1" expected_votes="1">
>                 <multicast addr="224.0.0.1"/>
>                 <multicast addr="224.0.0.9"/>
>         </cman>
>         <clusternodes>
>                 <clusternode name="atmail-1" nodeid="1" votes="1">
>                         <altname name="cluster1"/>
>                         <multicast addr="224.0.0.1" interface="eth0"/>
>                         <multicast addr="224.0.0.9" interface="eth1"/>
>                         <fence>
>                                 <method name="human">
>                                         <device name="last_resort" 
> ipaddr="clust
> er1"/>
>                                 </method>
>                         </fence>
>                 </clusternode>
>                 <clusternode name="atmail-2" nodeid="2" votes="1">
>                         <altname name="cluster2"/>
>                         <multicast addr="224.0.0.1" interface="eth0"/>
>                         <multicast addr="224.0.0.9" interface="eth1"/>
>                         <fence>
>                                 <method name="human">
>                                         <device name="last_resort" 
> ipaddr="clust
> er2"/>
>                                 </method>
>                         </fence>
>                 </clusternode>
>         </clusternodes>
>         <fencedevices>
>                 <fencedevice name="human" agent="fence_manual"/>
>         </fencedevices>
>         <fence_daemon clean_start="0">
>         </fence_daemon>
>  
> </cluster>
> #######################################################
>  
>  From the log file in node #1 already quorated I have:
>  
> Dec 16 15:29:38 atmail-1 kernel: CMAN: Waiting to join or form a 
> Linux-cluster
> Dec 16 15:29:39 atmail-1 ccsd[3191]: Connected to cluster infrastruture 
> via: CMA
> N/SM Plugin v1.1
> Dec 16 15:29:39 atmail-1 ccsd[3191]: Initial status:: Inquorate
> Dec 16 15:30:10 atmail-1 kernel: CMAN: forming a new cluster
> Dec 16 15:30:10 atmail-1 kernel: CMAN: quorum regained, resuming activity
> Dec 16 15:30:10 atmail-1 ccsd[3191]: Cluster is quorate.  Allowing 
> connections.
>  From the second node, inquorated we have:
>  
> Dec 16 15:30:22 atmail-2 ccsd[3260]: Remote copy of cluster.conf is from 
> quorate node.
> Dec 16 15:30:22 atmail-2 ccsd[3260]:  Local version # : 1
> Dec 16 15:30:22 atmail-2 ccsd[3260]:  Remote version #: 2
> Dec 16 15:30:22 atmail-2 ccsd[3260]: Switching to remote copy.
> Dec 16 15:30:25 atmail-2 kernel: CMAN: Waiting to join or form a 
> Linux-cluster
> Dec 16 15:30:25 atmail-2 ccsd[3260]: Connected to cluster infrastruture 
> via: CMAN/SM Plugin v1.1
> Dec 16 15:30:25 atmail-2 ccsd[3260]: Initial status:: Inquorate
> Dec 16 15:30:28 atmail-2 kernel: CMAN: sending membership request
> Dec 16 15:31:03 atmail-2 last message repeated 21 times
> Dec 16 15:32:08 atmail-2 last message repeated 39 times
> Dec 16 15:33:13 atmail-2 last message repeated 39 times
>  
> Any hints ????
>  
> We are using kernel 2.6.9, and we have applied all the patches 
> availables for it

Check your routes.  By default, your multicast stuff will go out
whatever your default route is.  You may need to add a route to force
the multicast out the proper NIC, e.g.

	route add -net 224.0.0.0 netmask 255.0.0.0 dev eth1
----------------------------------------------------------------------
- Rick Stevens, Senior Systems Engineer     rstevens at vitalstream.com -
- VitalStream, Inc.                       http://www.vitalstream.com -
-                                                                    -
-     I.R.S.: We've got what it takes to take what you've got!       -
----------------------------------------------------------------------