<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"> <head> <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii"> <meta name=Generator content="Microsoft Word 12 (filtered medium)"> <style>  </style>  </head> <body lang=EN-US link=blue vlink=purple> <div class=Section1> For two node clusters there's a convenient workaround: crossover cable.<o:p></o:p> <o:p> </o:p> You'll need a spare Ethernet port but that's easier than getting certain switches to do multicast correctly. (At least in my experience.)<o:p></o:p> <o:p> </o:p> <div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'> <div> <div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'> From: linux-cluster-bounces@redhat.com [mailto:linux-cluster-bounces@redhat.com] On Behalf Of Jason_Henderson@Mitel.com Sent: Thursday, April 15, 2010 1:44 PM To: linux clustering Cc: linux-cluster@redhat.com; linux-cluster-bounces@redhat.com Subject: Re: [Linux-cluster] Two node cluster,start CMAN fence the other node<o:p></o:p> </div> </div> <o:p> </o:p> Most likely the multicast packet communication between the 2 nodes is not getting through your network. <tt>linux-cluster-bounces@redhat.com wrote on 04/15/2010 01:05:01 PM:</tt> <tt>> Good afternoon,</tt> <tt>> I'm trying to form my first cluster of two nodes, using iLO fence </tt> <tt>> devices. I need some help because I can't find what I've missed. </tt> <tt>> My main problem is that the "service cman start" reboots the other </tt> <tt>> node and I can't form the two nodes cluster.</tt> <tt>> I'm using (at both nodea and nodeb, they are on the same VLAN and </tt> <tt>> pings each other ok):</tt> <tt>> </tt> <tt>> [root@nodea ~]# uname -a</tt> <tt>> Linux nodea 2.6.18-164.15.1.el5 #1 SMP Wed Mar 17 11:30:06 EDT 2010 </tt> <tt>> x86_64 x86_64 x86_64 GNU/Linux</tt> <tt>> [root@nodea ~]# rpm -qa |grep cman</tt> <tt>> cman-2.0.115-1.el5_4.9</tt> <tt>> </tt> <tt>> [root@nodea ~]# cat /etc/cluster/cluster.conf (nodeb has the same file)</tt> <tt>> <?xml version="1.0" ?></tt> <tt>> <cluster alias="VCluster" config_version="5" name="VCluster"></tt> <tt>> <fence_daemon post_fail_delay="0" post_join_delay="25"/></tt> <tt>> <clusternodes></tt> <tt>> <clusternode name="nodea" nodeid="1" votes="1"></tt> <tt>> <fence></tt> <tt>> <method name="1"></tt> <tt>> <device name="nodeaILO"/></tt> <tt>> </method></tt> <tt>> </fence></tt> <tt>> </clusternode></tt> <tt>> <clusternode name="nodeb" nodeid="2" votes="1"></tt> <tt>> <fence></tt> <tt>> <method name="1"></tt> <tt>> <device name="nodebILO"/></tt> <tt>> </method></tt> <tt>> </fence></tt> <tt>> </clusternode></tt> <tt>> </clusternodes></tt> <tt>> <cman expected_votes="1" two_node="1"/></tt> <tt>> <fencedevices></tt> <tt>> <fencedevice agent="fence_ilo" hostname="nodeacn" </tt> <tt>> login="user" name="nodeaILO" passwd="hp"/></tt> <tt>> <fencedevice agent="fence_ilo" hostname="nodebcn" </tt> <tt>> login="user" name="nodebILO" passwd="hp"/></tt> <tt>> </fencedevices></tt> <tt>> <rm></tt> <tt>> <failoverdomains/></tt> <tt>> <resources/></tt> <tt>> </rm></tt> <tt>> </cluster></tt> <tt>> </tt> <tt>> When I start the cman service, it hangs up for some time at the </tt> <tt>> "Starting fencing..." step and after those configured 25secs it </tt> <tt>> fences nodeb and reboots it.</tt> <tt>> [root@nodea ~]# service cman start</tt> <tt>> Starting cluster: </tt> <tt>> Loading modules... done</tt> <tt>> Mounting configfs... done</tt> <tt>> Starting ccsd... done</tt> <tt>> Starting cman... done</tt> <tt>> Starting daemons... done</tt> <tt>> Starting fencing... done</tt> <tt>> [ OK ]</tt> <tt>> </tt> <tt>> "nodeb" gets rebooted:</tt> <tt>> [root@nodeb ~]# </tt> <tt>> Broadcast message from root (Thu Apr 15 18:42:24 2010):</tt> <tt>> </tt> <tt>> The system is going down for system halt NOW!</tt> <tt>> </tt> <tt>> At the syslog I just can find:</tt> <tt>> Apr 15 18:40:59 nodea ccsd[16930]: Initial status:: Quorate </tt> <tt>> Apr 15 18:40:59 nodea openais[16936]: [CLM ] Members Left: </tt> <tt>> Apr 15 18:40:59 nodea openais[16936]: [CLM ] Members Joined: </tt> <tt>> Apr 15 18:40:59 nodea openais[16936]: [CLM ] CLM CONFIGURATION CHANGE </tt> <tt>> Apr 15 18:41:00 nodea openais[16936]: [CLM ] New Configuration: </tt> <tt>> Apr 15 18:41:00 nodea openais[16936]: [CLM ] r(0) ip(10.192.16.42) </tt> <tt>> Apr 15 18:41:00 nodea openais[16936]: [CLM ] Members Left: </tt> <tt>> Apr 15 18:41:00 nodea openais[16936]: [CLM ] Members Joined: </tt> <tt>> Apr 15 18:41:00 nodea openais[16936]: [CLM ] r(0) ip(10.192.16.42) </tt> <tt>> Apr 15 18:41:00 nodea openais[16936]: [SYNC ] This node is within </tt> <tt>> the primary component and will provide service. </tt> <tt>> Apr 15 18:41:00 nodea openais[16936]: [TOTEM] entering OPERATIONAL state. </tt> <tt>> Apr 15 18:41:00 nodea openais[16936]: [CMAN ] quorum regained, </tt> <tt>> resuming activity </tt> <tt>> Apr 15 18:41:00 nodea openais[16936]: [CLM ] got nodejoin message </tt> <tt>> 10.192.16.42 </tt> <tt>> Apr 15 18:42:11 nodea fenced[16955]: nodeb not a cluster member </tt> <tt>> after 25 sec post_join_delay</tt> <tt>> Apr 15 18:42:11 nodea fenced[16955]: fencing node "nodeb"</tt> <tt>> Apr 15 18:42:23 nodea fenced[16955]: fence "nodeb" success</tt> <tt>> </tt> <tt>> [root@nodea ~]# clustat</tt> <tt>> Cluster Status for VCluster @ Thu Apr 15 18:55:23 2010</tt> <tt>> Member Status: Quorate</tt> <tt>> </tt> <tt>> Member Name ID Status</tt> <tt>> ------ ---- ---- ------</tt> <tt>> nodea </tt> <tt>> 1 Online, Local</tt> <tt>> nodeb 2 Offline</tt> <tt>> </tt> <tt>> Then when nodeb starts again, I try to start cman there to join the </tt> <tt>> cluster... but it again fences "nodea":</tt> <tt>> [root@nodeb ~]# clustat</tt> <tt>> Could not connect to CMAN: No such file or directory</tt> <tt>> [root@nodeb ~]# service cman start</tt> <tt>> Starting cluster: </tt> <tt>> Loading modules... done</tt> <tt>> Mounting configfs... done</tt> <tt>> Starting ccsd... done</tt> <tt>> Starting cman... done</tt> <tt>> Starting qdiskd... done</tt> <tt>> Starting daemons... done</tt> <tt>> Starting fencing... (wait for 25secs again) done</tt> <tt>> [ OK ]</tt> <tt>> "nodea" gets rebooted:</tt> <tt>> [root@nodea ~]# </tt> <tt>> Broadcast message from root (Thu Apr 15 18:58:40 2010):</tt> <tt>> </tt> <tt>> The system is going down for system halt NOW!</tt> <tt>> </tt> <tt>> Apr 15 18:57:31 nodeb openais[11789]: [CLM ] Members Joined: </tt> <tt>> Apr 15 18:57:31 nodeb openais[11789]: [CLM ] r(0) ip(10.192.16.44) </tt> <tt>> Apr 15 18:57:31 nodeb openais[11789]: [SYNC ] This node is within </tt> <tt>> the primary component and will provide service. </tt> <tt>> Apr 15 18:57:31 nodeb openais[11789]: [TOTEM] entering OPERATIONAL state. </tt> <tt>> Apr 15 18:57:31 nodeb openais[11789]: [CMAN ] quorum regained, </tt> <tt>> resuming activity </tt> <tt>> Apr 15 18:57:31 nodeb openais[11789]: [CLM ] got nodejoin message </tt> <tt>> 10.192.16.44 </tt> <tt>> Apr 15 18:57:34 nodeb qdiskd[10323]: <info> Quorum Daemon Initializing </tt> <tt>> Apr 15 18:57:34 nodeb qdiskd[10323]: <crit> Initialization failed </tt> <tt>> Apr 15 18:58:42 nodeb fenced[11816]: nodea not a cluster member </tt> <tt>> after 25 sec post_join_delay</tt> <tt>> Apr 15 18:58:42 nodeb fenced[11816]: fencing node "nodea"</tt> <tt>> Apr 15 18:58:54 nodeb fenced[11816]: fence "nodea" success</tt> <tt>> </tt> <tt>> And I can't get the two nodes, joining the cluster...</tt> <tt>> I guess I'm missing something at the cluster.conf file??? I can't </tt> <tt>> find what I'm making wrong.</tt> <tt>> </tt> <tt>> Thanks for any help!</tt> <tt>> </tt> <tt>> Alex Re--</tt> <tt>> Linux-cluster mailing list</tt> <tt>> Linux-cluster@redhat.com</tt> <tt>> https://www.redhat.com/mailman/listinfo/linux-cluster</tt><o:p></o:p> </div> </div> </body> </html>