[Linux-cluster] Shutdown/startup

[Paul Tader]

David Teigland wrote:
> On Thu, Mar 23, 2006 at 03:12:29PM -0600, Paul Tader wrote:
> 
>>So how does one properly shutdown and/or start up a GFS 6.1 cluster. 
>>Documentation on the procedures are surprisingly sparse.  For example, 
>>what I've experienced during shutdown are messages like:
>>
>>CMAN: sendmsg failed: -101
>>CMAN: we are leaving the cluster.
>>
>>from the node shutting down while the the other nodes report that this 
>>node needs to be fenced off.  What?!  Fenced during a controlled 
>>shutdown?  Is that correct?  During a complete cluster shutdown, of 
>>course quorum, at some point is lost, freezing the nodes that have not 
>>completely shutdown forcing a hard reset on those nodes.  I'm probably 
>>missing something trivial here, but this doesn't seem right.
>>
>>Start up also suffers from the "quorum" problem, but it seems to 
>>recovery once enough nodes are up.
> 
> 
> You read the notes here about startup/shutdown?
> http://sources.redhat.com/cluster/doc/usage.txt
> 
> To shutdown, you umount all fs's on all nodes, fence_tool leave on
> all nodes, then leave the cluster on all.  To leave the cluster without
> quorum getting in the way use 'cman_tool leave remove'.
> 
> To start up, have all nodes join the cluster with cman_tool join,
> then have all nodes join the fence domain with fence_tool join,
> then start other things.
> 
> Dave
> 
> 

Thanks Dave.  The process detailed in this document brought the nodes 
down gracefully.

[root at node]# umount /gfs_volume
[root at node]# vgchange -aln
[root at node]# killall clvmd
[root at node]# fence_tool leave
[root at node]# cman_tool leave -w
[root at node]# killall ccsd
[root at node]# (reboot/shutdown)


I'm going to incorporate this in to a revised shutdown init script and 
post what work.

Paul


-- 
===========================================================================
Paul Tader  <ptader at fnal.gov> Computing Div/CSS Dept
Fermi National Accelerator Lab; PO Box 500 MS 369 Batavia, IL 60510-0500