[Linux-cluster] clvmd startup timed out

[Kadlecsik Jozsi]

Hi,

In our GFS cluster clvmd does not start up sometimes. It seems the stuck 
clmvd cannot recover, even when the node is a cluster member and the 
quorum is OK:

root at lxserv0:~# cman_tool status
Version: 6.0.1
Config Version: 6
Cluster Name: kfki
Cluster Id: 1583
Cluster Member: Yes
Cluster Generation: 516
Membership state: Cluster-Member
Nodes: 4
Expected votes: 5
Total votes: 4
Quorum: 3  
Active subsystems: 7
Flags: 
Ports Bound: 0 11  
Node name: lxserv0-gfs
Node ID: 1
Multicast addresses: 224.0.0.3 
Node addresses: 192.168.192.15 

root at lxserv0:~# cman_tool services
type             level name     id       state       
fence            0     default  00010001 none        
[1 2 3 4]
dlm              1     clvmd    00010002 none        
[1 2 3 4]

In consequence, 'vgchange -aly' hangs in the init script and we cannot 
proceed to mount the GFS volumes.

When starting clvmd in debug mode, all I get is

root at lxserv0:~# /sbin/clvmd -T 10 -d
CLVMD[b7d7d6c0]: Aug 29 14:57:16 CLVMD started
CLVMD[b7d7d6c0]: Aug 29 14:57:16 Connected to CMAN
CLVMD[b7d7d6c0]: Aug 29 14:57:19 CMAN initialisation complete

Nothing more and vgchange still hangs.

What is missing here?

Best regards,
Jozsef
--
E-mail : kadlec at sunserv.kfki.hu, kadlec at blackhole.kfki.hu
PGP key: http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address: KFKI Research Institute for Particle and Nuclear Physics
         H-1525 Budapest 114, POB. 49, Hungary