<html><body><div style="color:#000; background-color:#fff; font-family:times new roman, new york, times, serif;font-size:12pt"><div>Hi All,</div><div> </div><div>This is Junaid. I have two node RHEL 5.3 cluster servers. These are oracle database servers. Unfortunately, there was a power outage in datacenter. One of the active server was rebooted but failover did not happen to other server.</div><div> </div><div>I am investigating to find root cause but still no evidence.</div><div> </div><div>Please help me to find out root cause as I need to report my manager asap.</div><div> </div><div>Here is cluster log error:</div><div> </div><div>Nov 4 14:05:55 akarsum1 openais[5827]: [TOTEM] Token Timeout (10000 ms) retransmit timeout (495 ms)<br>Nov 4 14:05:55 akarsum1 openais[5827]: [TOTEM] token hold (386 ms) retransmits before loss (20 retrans)<br>Nov 4 14:05:55 akarsum1 openais[5827]: [TOTEM] join (60 ms)
send_join (0 ms) consensus (4800 ms) merge (200 ms)<br>Nov 4 14:05:55 akarsum1 openais[5827]: [TOTEM] downcheck (1000 ms) fail to recv const (50 msgs)<br>Nov 4 14:05:55 akarsum1 openais[5827]: [TOTEM] seqno unchanged const (30 rotations) Maximum network MTU 1500<br>Nov 4 14:05:55 akarsum1 openais[5827]: [TOTEM] window size per rotation (50 messages) maximum messages per rotation (17 messages)<br>Nov 4 14:05:55 akarsum1 openais[5827]: [TOTEM] send threads (0 threads)<br>Nov 4 14:05:55 akarsum1 openais[5827]: [TOTEM] RRP token expired timeout (495 ms)<br>Nov 4 14:05:55 akarsum1 openais[5827]: [TOTEM] RRP token problem counter (2000 ms)<br>Nov 4 14:05:55 akarsum1 openais[5827]: [TOTEM] RRP threshold (10 problem count)<br>Nov 4 14:05:55 akarsum1 openais[5827]: [TOTEM] RRP mode set to none.<br>Nov 4 14:05:55 akarsum1 openais[5827]: [TOTEM] heartbeat_failures_allowed (0)<br>Nov 4 14:05:55 akarsum1
openais[5827]: [TOTEM] max_network_delay (50 ms)<br>Nov 4 14:05:55 akarsum1 openais[5827]: [TOTEM] HeartBeat is Disabled. To enable set heartbeat_failures_allowed > 0<br>Nov 4 14:05:55 akarsum1 openais[5827]: [TOTEM] Receive multicast socket recv buffer size (288000 bytes).<br>Nov 4 14:05:55 akarsum1 openais[5827]: [TOTEM] Transmit multicast socket send buffer size (288000 bytes).<br>Nov 4 14:05:55 akarsum1 openais[5827]: [TOTEM] The network interface [10.128.7.13] is now up.<br>Nov 4 14:05:55 akarsum1 openais[5827]: [TOTEM] Created or loaded sequence id 4728.10.128.7.13 for this ring.<br>Nov 4 14:05:55 a<var id="yui-ie-cursor"></var>karsum1 openais[5827]: [TOTEM] entering GATHER state from 15.<br>Nov 4 14:05:55 akarsum1 openais[5827]: [SERV ] Initialising service handler 'openais extended virtual synchrony service'<br>Nov 4 14:05:55 akarsum1 openais[5827]: [SERV ] Initialising service handler 'openais
cluster membership service B.01.01'<br>Nov 4 14:05:55 akarsum1 openais[5827]: [SERV ] Initialising service handler 'openais availability management framework B.01.01'<br>Nov 4 14:05:55 akarsum1 openais[5827]: [SERV ] Initialising service handler 'openais checkpoint service B.01.01'<br>Nov 4 14:05:55 akarsum1 openais[5827]: [SERV ] Initialising service handler 'openais event service B.01.01'<br>Nov 4 14:05:55 akarsum1 openais[5827]: [SERV ] Initialising service handler 'openais distributed locking service B.01.01'<br>Nov 4 14:05:55 akarsum1 openais[5827]: [SERV ] Initialising service handler 'openais message service B.01.01'<br>Nov 4 14:05:55 akarsum1 openais[5827]: [SERV ] Initialising service handler 'openais configuration service'<br></div><div> </div><div> </div></div></body></html>