[Linux-cluster] failover problem

[Junaid Khan]

Hi All,
 
This is Junaid. I have two node RHEL 5.3 cluster servers. These are oracle database servers. Unfortunately, there was a power outage in datacenter. One of the active server was rebooted but failover did not happen to other server.
 
I am investigating to find root cause but still no evidence.
 
Please help me to find out root cause as I need to report my manager asap.
 
Here is cluster log error:
 
Nov  4 14:05:55 akarsum1 openais[5827]: [TOTEM] Token Timeout (10000 ms) retransmit timeout (495 ms)
Nov  4 14:05:55 akarsum1 openais[5827]: [TOTEM] token hold (386 ms) retransmits before loss (20 retrans)
Nov  4 14:05:55 akarsum1 openais[5827]: [TOTEM] join (60 ms) send_join (0 ms) consensus (4800 ms) merge (200 ms)
Nov  4 14:05:55 akarsum1 openais[5827]: [TOTEM] downcheck (1000 ms) fail to recv const (50 msgs)
Nov  4 14:05:55 akarsum1 openais[5827]: [TOTEM] seqno unchanged const (30 rotations) Maximum network MTU 1500
Nov  4 14:05:55 akarsum1 openais[5827]: [TOTEM] window size per rotation (50 messages) maximum messages per rotation (17 messages)
Nov  4 14:05:55 akarsum1 openais[5827]: [TOTEM] send threads (0 threads)
Nov  4 14:05:55 akarsum1 openais[5827]: [TOTEM] RRP token expired timeout (495 ms)
Nov  4 14:05:55 akarsum1 openais[5827]: [TOTEM] RRP token problem counter (2000 ms)
Nov  4 14:05:55 akarsum1 openais[5827]: [TOTEM] RRP threshold (10 problem count)
Nov  4 14:05:55 akarsum1 openais[5827]: [TOTEM] RRP mode set to none.
Nov  4 14:05:55 akarsum1 openais[5827]: [TOTEM] heartbeat_failures_allowed (0)
Nov  4 14:05:55 akarsum1 openais[5827]: [TOTEM] max_network_delay (50 ms)
Nov  4 14:05:55 akarsum1 openais[5827]: [TOTEM] HeartBeat is Disabled. To enable set heartbeat_failures_allowed > 0
Nov  4 14:05:55 akarsum1 openais[5827]: [TOTEM] Receive multicast socket recv buffer size (288000 bytes).
Nov  4 14:05:55 akarsum1 openais[5827]: [TOTEM] Transmit multicast socket send buffer size (288000 bytes).
Nov  4 14:05:55 akarsum1 openais[5827]: [TOTEM] The network interface [10.128.7.13] is now up.
Nov  4 14:05:55 akarsum1 openais[5827]: [TOTEM] Created or loaded sequence id 4728.10.128.7.13 for this ring.
Nov  4 14:05:55 akarsum1 openais[5827]: [TOTEM] entering GATHER state from 15.
Nov  4 14:05:55 akarsum1 openais[5827]: [SERV ] Initialising service handler 'openais extended virtual synchrony service'
Nov  4 14:05:55 akarsum1 openais[5827]: [SERV ] Initialising service handler 'openais cluster membership service B.01.01'
Nov  4 14:05:55 akarsum1 openais[5827]: [SERV ] Initialising service handler 'openais availability management framework B.01.01'
Nov  4 14:05:55 akarsum1 openais[5827]: [SERV ] Initialising service handler 'openais checkpoint service B.01.01'
Nov  4 14:05:55 akarsum1 openais[5827]: [SERV ] Initialising service handler 'openais event service B.01.01'
Nov  4 14:05:55 akarsum1 openais[5827]: [SERV ] Initialising service handler 'openais distributed locking service B.01.01'
Nov  4 14:05:55 akarsum1 openais[5827]: [SERV ] Initialising service handler 'openais message service B.01.01'
Nov  4 14:05:55 akarsum1 openais[5827]: [SERV ] Initialising service handler 'openais configuration service'
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/linux-cluster/attachments/20111116/bd3813e7/attachment.htm>