[Linux-cluster] Node will not join cluster.

[Danny Wall]

I have a two node cluster on RHEL4. It was all Update 4, but Red Hat suggested I update the problem node to U5, so now it is mixed.

The problem:
We had a SAN issue last week that took 3 of the 15 LUNs offline. The SAN problem was been fixed, so I tried to rescan the LUNs, and still could not see the LUNs. I restarted one node, and ever since then, it would fail when trying to join the cluster. I disabled the cluster services at boot, and try them manually. Every time I get to CLVMD, it hangs forever. Sometimes I can CTRL-C to break out, sometimes no. There are no relevant messages in /var/log/messages. I have worked with Red Hat and the last thing they suggested was to upgrade the node to U5. I did Tuesday, and it still hangs. I am afraid to take down the other node, because if it does the same thing, I am going to have major problems.

I have also noticed that running lvdisplay, vgdisplay and pvdisplay hang. I am sure it is related. I know dlm and module is loaded.

lock_dlm               47220  0
lock_harness            7216  2 gfs,lock_dlm
dlm                   134404  3 lock_dlm
cman                  139552  11 gfs,lock_dlm,dlm


Any thoughts?

Danny


#####################################
This message is for the named person's use only.  It may
contain confidential, proprietary, or legally privileged
information.  No confidentiality or privilege is waived or
lost by any mistransmission.  If you receive this message
in error, please immediately delete it and all copies of it
from your system, destroy any hard copies of it, and notify
the sender.  You must not, directly or indirectly, use,
disclose, distribute, print, or copy any part of this message
if you are not the intended recipient.  Health First reserves
the right to monitor all e-mail communications through its
networks.  Any views or opinions expressed in this message
are solely those of the individual sender, except (1) where
the message states such views or opinions are on behalf of
a particular entity;  and (2) the sender is authorized by
the entity to give such views or opinions.
#####################################