<html>
<head>
<style>
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
FONT-SIZE: 10pt;
FONT-FAMILY:Tahoma
}
</style>
</head>
<body class='hmmessage'>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB style="FONT-SIZE: 10pt; FONT-FAMILY: Arial; mso-ansi-language: EN-GB"><FONT color=#000000>Hi</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB style="FONT-SIZE: 10pt; FONT-FAMILY: Arial; mso-ansi-language: EN-GB"><FONT color=#000000></FONT></SPAN> </P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB style="FONT-SIZE: 10pt; FONT-FAMILY: Arial; mso-ansi-language: EN-GB"><FONT color=#000000> I’m having fencing problems in my 3 node cluster running on RHEL5.0 which involves bonding.<?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" /><o:p></o:p></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB style="FONT-SIZE: 10pt; FONT-FAMILY: Arial; mso-ansi-language: EN-GB"><o:p><FONT color=#000000> </FONT></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB style="FONT-SIZE: 10pt; FONT-FAMILY: Arial; mso-ansi-language: EN-GB"><FONT color=#000000>I have 3 severs A, B & C in a cluster<SPAN style="mso-spacerun: yes"> </SPAN>with bonding configured on eth2 & eth3 for my cluster traffic. <SPAN style="mso-spacerun: yes"> </SPAN>The config is as below:</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB style="FONT-SIZE: 10pt; FONT-FAMILY: Arial; mso-ansi-language: EN-GB"><FONT color=#000000></FONT></SPAN> </P><SPAN lang=EN-GB style="FONT-SIZE: 10pt; FONT-FAMILY: Arial; mso-ansi-language: EN-GB">
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>DEVICE=eth2</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>BOOTPROTO=none</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>ONBOOT=yes</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>TYPE=Ethernet</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>MASTER=bond1</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>SLAVE=yes</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>USRCTL=no</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><o:p><FONT face="Times New Roman" color=#000000 size=3> </FONT></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>DEVICE=eth3</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>BOOTPROTO=none</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>ONBOOT=yes</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>TYPE=Ethernet</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>MASTER=bond1</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>SLAVE=yes</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>USRCTL=no</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"> </P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"> </P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt">DEVICE=bond1</P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt">IPADDR=192.168.x.x</P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt">NETMASK=255.255.255.0</P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt">NETWORK=192.168.x.0</P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt">BROADCAST=192.168.x.255</P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt">ONBOOT=YES</P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt">BOOTPROTO=none</P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"> </P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>The /etc/modprobe.conf file is configured as below:</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><o:p><FONT face="Times New Roman" color=#000000 size=3> </FONT></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>alias eth0 bnx2</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>alias eth1 bnx2</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>alias eth2 e1000</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>alias eth3 e1000</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>alias eth4 e1000</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>alias eth5 e1000</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>alias scsi_hostadapter cciss</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>alias bond0 bonding</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>options bond0 miimon=100 mode=active-backup max_bonds=3</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>alias bond1 bonding</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>options bond1 miimon=100 mode=active-backup</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>alias bond2 bonding</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>options bond2 miimon=100 mode=active-backup</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=SV style="mso-ansi-language: SV"><FONT size=3><FONT color=#000000><FONT face="Times New Roman">alias scsi_hostadapter1 qla2xxx<o:p></o:p></FONT></FONT></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=SV style="mso-ansi-language: SV"><FONT size=3><FONT color=#000000><FONT face="Times New Roman">alias scsi_hostadapter2 usb-storage<o:p></o:p></FONT></FONT></FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=SV style="mso-ansi-language: SV"><o:p><FONT face="Times New Roman" color=#000000 size=3> </FONT></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=SV style="mso-ansi-language: SV"><o:p><FONT face="Times New Roman" color=#000000 size=3> </FONT></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>The cluster starts up OK, however when I try to test the bonded interfaces my troubles begin.</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>On Node C if I "ifdown bond1", the node C, is fenced and everything works as expected.</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB></SPAN> </P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>However if on Node C, I take down the interfaces one at a time i.e. </FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3> "ifdown<SPAN style="mso-spacerun: yes"> </SPAN>eth2", - the cluster stays up as expected using eth3 for routing traffic </FONT></SPAN><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3> </FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3> "ifdown eth3" </FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>then node C is fenced by Node A. However in the /var/log/messages file on Node C I see a message saying that Node B will be fenced. The outcome is Nodes C & B are fenced. </FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><o:p><FONT face="Times New Roman" color=#000000 size=3> </FONT></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3>My question is why does node B get fenced as well?</FONT></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><FONT face="Times New Roman" color=#000000 size=3></FONT></SPAN> </P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB></SPAN> </P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"><SPAN lang=EN-GB><o:p><FONT face="Times New Roman" color=#000000 size=3> D.</FONT></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"> </P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"> </P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"> </P>
<P class=MsoNormal style="MARGIN: 0cm 0cm 0pt"></SPAN> </P></P><br /><hr />Get fish-slapping on Messenger! <a href='http://clk.atdmt.com/UKM/go/msnnkmgl0010000008ukm/direct/01/' target='_new'>Play Now</a></body>
</html>