[Linux-cluster] Testing cluster failover
KC LO
kclo2000 at gmail.com
Fri Jan 14 04:39:37 UTC 2011
Dear Friends,
Thanks for your advise on my new cluster setup. I am going into failover
testing of the cluster.
My configuraiton involves 3 active member and 1 standby servers and running
Redhat 5.5 with Cluster installed.
In my active server, when I type "reboot", it can successfully relocate the
service to the standby node.
However, If I type "init 0" to simulate server failure, it can't relocate to
the standby node. Is it the correct behaviour?
Anything that I should check. Any advice?
Thanks for your help!
Below is the rgmanager log file from the standby server. It can detect the
server down.
*Jan 14 11:24:53 yzbstb01 clurgmgrd[16286]: <notice> Member 1 shutting down
*Jan 14 11:25:17 yzbstb01 openais[15365]: [TOTEM] The token was lost in the
OPERATIONAL state.
Jan 14 11:25:17 yzbstb01 openais[15365]: [TOTEM] Receive multicast socket
recv buffer size (320000 bytes).
Jan 14 11:25:17 yzbstb01 openais[15365]: [TOTEM] Transmit multicast socket
send buffer size (262142 bytes).
Jan 14 11:25:17 yzbstb01 openais[15365]: [TOTEM] entering GATHER state from
2.
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] entering GATHER state from
0.
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] Saving state aru c8 high
seq received c8
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] Storing new sequence id for
ring 5e8
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] entering COMMIT state.
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] entering RECOVERY state.
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] position [0] member
10.10.10.164:
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1508 rep
10.10.10.164
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] aru c8 high delivered c8
received flag 1
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] position [1] member
10.10.10.165:
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1508 rep
10.10.10.164
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] aru c8 high delivered c8
received flag 1
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] position [2] member
10.10.10.167:
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1508 rep
10.10.10.164
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] aru c8 high delivered c8
received flag 1
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] Did not need to originate
any messages in recovery.
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] CLM CONFIGURATION CHANGE
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] New Configuration:
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0)
ip(10.10.10.164)
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0)
ip(10.10.10.165)
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0)
ip(10.10.10.167)
*Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] Members Left:
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0)
ip(10.10.10.166)
*Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] Members Joined:
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] CLM CONFIGURATION CHANGE
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] New Configuration:
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0)
ip(10.10.10.164)
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0)
ip(10.10.10.165)
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0)
ip(10.10.10.167)
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] Members Left:
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] Members Joined:
Jan 14 11:25:37 yzbstb01 openais[15365]: [SYNC ] This node is within the
primary component and will provide service.
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] entering OPERATIONAL state.
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] got nodejoin message
10.10.10.164
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] got nodejoin message
10.10.10.165
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] got nodejoin message
10.10.10.167
Jan 14 11:25:37 yzbstb01 openais[15365]: [CPG ] got joinlist message from
node 4
Jan 14 11:25:37 yzbstb01 openais[15365]: [CPG ] got joinlist message from
node 3
Jan 14 11:25:37 yzbstb01 openais[15365]: [CPG ] got joinlist message from
node 2
Jan 14 11:51:34 yzbstb01 openais[15365]: [TOTEM] entering GATHER state from
9.
Jan 14 11:51:37 yzbstb01 openais[15365]: [TOTEM] Saving state aru 25 high
seq received 25
Jan 14 11:51:37 yzbstb01 openais[15365]: [TOTEM] Storing new sequence id for
ring 5ec
Jan 14 11:51:37 yzbstb01 openais[15365]: [TOTEM] entering COMMIT state.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] The token was lost in the
COMMIT state.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] entering GATHER state from
4.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] Storing new sequence id for
ring 5f0
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] entering COMMIT state.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] entering GATHER state from
13.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] Storing new sequence id for
ring 5f4
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] entering COMMIT state.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] entering RECOVERY state.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] position [0] member
10.10.10.164:
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1512 rep
10.10.10.164
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] aru 25 high delivered 25
received flag 1
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] position [1] member
10.10.10.165:
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1512 rep
10.10.10.164
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] aru 25 high delivered 25
received flag 1
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] position [2] member
10.10.10.166:
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1512 rep
10.10.10.166
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] aru c high delivered c
received flag 1
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] position [3] member
10.10.10.167:
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1512 rep
10.10.10.164
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] aru 25 high delivered 25
received flag 1
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] Did not need to originate
any messages in recovery.
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] CLM CONFIGURATION CHANGE
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] New Configuration:
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0)
ip(10.10.10.164)
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0)
ip(10.10.10.165)
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0)
ip(10.10.10.167)
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] Members Left:
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] Members Joined:
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] CLM CONFIGURATION CHANGE
*Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] New Configuration:
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0)
ip(10.10.10.164)
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0)
ip(10.10.10.165)
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0)
ip(10.10.10.166)
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0)
ip(10.10.10.167)
*Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] Members Left:
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] Members Joined:
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0)
ip(10.10.10.166)
Jan 14 11:51:47 yzbstb01 openais[15365]: [SYNC ] This node is within the
primary component and will provide service.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] entering OPERATIONAL state.
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] got nodejoin message
10.10.10.165
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] got nodejoin message
10.10.10.166
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] got nodejoin message
10.10.10.167
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] got nodejoin message
10.10.10.164
Jan 14 11:51:47 yzbstb01 openais[15365]: [CPG ] got joinlist message from
node 3
Jan 14 11:51:47 yzbstb01 openais[15365]: [CPG ] got joinlist message from
node 1
Jan 14 11:51:47 yzbstb01 openais[15365]: [CPG ] got joinlist message from
node 2
Jan 14 11:51:47 yzbstb01 openais[15365]: [CPG ] got joinlist message from
node 4
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/linux-cluster/attachments/20110114/a7306c44/attachment.htm>
More information about the Linux-cluster
mailing list