[Linux-cluster] ccsd problems after update to RHEL 5.2/5.3

[Gordan Bobic]

I have a two-node cluster and ever since I updated the kernel and 
cluster components I cannot get more than one node running with GFS.

Here are the package versions I have:
kernel-2.6.18-92.1.22.el5
cman-2.0.98-1
kmod-gfs-0.1.23-5.el5_2.4
gfs-utils-0.1.17-1.el5
gfs2-utils-0.1.53-1.1

Node 2 starts up OK, but I see this in the syslog:

node2 ccsd[5897]: Unable to perform sendto: Cannot assign requested address

When I power up node2, it just gets strange and the whole thing locks up:
node2 openais[5941]: [CMAN ] cman killed by node 1 because we rejoined 
the cluster without a full restart
node2 groupd[5953]: cman_get_nodes error -1 104
node2 gfs_controld[5995]: groupd_dispatch error -1 errno 11
node2 gfs_controld[5995]: groupd connection died
node2 gfs_controld[5995]: cluster is down, exiting

So for some reason node 1's joining makes node 2 get kicked out of the 
cluster - but worse, it doesn't seem to initiate fencing. Instead, the 
whole cluster just locks up on GFS access.

What am I missing? What should I be looking for in the logs? This 
cluster worked fine before the update.

I found this:
http://rhn.redhat.com/errata/RHBA-2009-0189.html
but updating cman to 2.0.98 as per the RHBA didn't fix the problem.

Gordan