[Linux-cluster] Initiating transition message

[Christine Caulfield]

Shawn Hood wrote:
> Though one instance of 'Initating transition' message seems to be
> normal , what could the behavior shown in the following log indicate?
> What exactly is happening during an 'Initating transition' message?
> 
> Shawn
> 
> Feb 14 15:25:55 odin kernel: CMAN: Initiating transition, generation 7
> Feb 14 15:26:01 odin kernel: CMAN: removing node munin from the
> cluster : No response to messages
> Feb 14 15:26:01 odin kernel: CMAN: Initiating transition, generation 8
> Feb 14 15:26:16 odin kernel: CMAN: Initiating transition, generation 9
> Feb 14 15:26:31 odin kernel: CMAN: Initiating transition, generation 10
> Feb 14 15:26:40 odin su(pam_unix)[20082]: session opened for user root
> by shood(uid=0)
> Feb 14 15:26:46 odin kernel: CMAN: Initiating transition, generation 11
> Feb 14 15:27:01 odin kernel: CMAN: Initiating transition, generation 12
> Feb 14 15:27:16 odin kernel: CMAN: Initiating transition, generation 13
> Feb 14 15:27:31 odin kernel: CMAN: Initiating transition, generation 14
> Feb 14 15:27:46 odin kernel: CMAN: Initiating transition, generation 15
> Feb 14 15:28:01 odin kernel: CMAN: Initiating transition, generation 16
> Feb 14 15:28:16 odin kernel: CMAN: Initiating transition, generation 17
> Feb 14 15:28:31 odin kernel: CMAN: Initiating transition, generation 18
> Feb 14 15:28:46 odin kernel: CMAN: too many transition restarts - will die
> Feb 14 15:28:46 odin kernel: CMAN: we are leaving the cluster.
> Inconsistent cluster view
> Feb 14 15:28:46 odin kernel: WARNING: dlm_emergency_shutdown
> Feb 14 15:28:46 odin kernel: WARNING: dlm_emergency_shutdown
> Feb 14 15:28:46 odin kernel: SM: 00000002 sm_stop: SG still joined
> Feb 14 15:28:46 odin kernel: SM: 01000004 sm_stop: SG still joined
> Feb 14 15:28:46 odin kernel: SM: 02000014 sm_stop: SG still joined
> Feb 14 15:28:46 odin ccsd[17392]: Cluster manager shutdown.  Attemping


The usual cause of all those messages (not that it's usual!) is network
problems. Often a one-way connection can cause it, eg the node can send
messages but not receive them. There are pathological iptables rules
that can make that happen too.

It's hard to be specific without knowing more, but I would investigate
the network connections, routers/switches and routing/iptables rules

Chrissie