[Linux-cluster] FAQ correction ?!

[Lon Hohberger]

On Tue, 2007-03-13 at 12:41 -0500, Robert Peterson wrote:
> Fajar Priyanto wrote:
> > On Tuesday 13 March 2007 16:49, Hugues Lafarge wrote:
> >> I'm not sure i correctly understand
> >> http://sources.redhat.com/cluster/faq.html#cman_2to3 so i'm kindly asking
> >> you if there is not a typo in the following sentence:
> >>
> >> 8. How do I add a third node to my two-node cluster?
> >>
> >>  Unfortunately, two-node clusters are a special case. Because a
> >>  two-node cluster only needs two nodes to establish quorum, the only
> >>  way to add a third node is to shut down the cluster, add the third
> >>  node into your /etc/cluster/cluster.conf file and get rid of
> >>  two_node="1", then restart all three nodes.
> >>
> >>
> >> I would have expected that to be:
> >>
> >>  Unfortunately, two-node clusters are a special case. Because a
> >>  two-node cluster only needs ONE NODE to establish quorum, ...
> >>
> >> Am i right there, or misunderstanding something ?
> > 
> > In my opinion, you're both correct.
> > But, this sentence can be easier to understand:
> > Two-node clusters are a special case. Because a
> > two-node cluster only needs two nodes to establish quorum, the only
> > way to add a third node is to shut down the cluster, add the third
> > node into your /etc/cluster/cluster.conf file (either by hand or 
> > system-config-cluster), copy it into the newly joined cluster, and restart 
> > all of the nodes. (the two_node="1" will be removed automatically by 
> > system-config-cluster).
> 
> Hi,
> 
> Actually, in a two-node cluster (without quorum disk/partition)
> two nodes are necessary to establish quorum, but only one node
> is enough to maintain quorum once it's established.

Actually, that's not true.  With two_node="1", cman will fence the node
that isn't a cluster member in order to establish itself as the only
node alive.  If the two nodes can not see each other but are both online
(e.g. network partition), this situation is called the 'race-to-fence'
condition.

Under normal circumstances, one node can not communicate - and therefore
- cannot fence the other node. 

(For RHCS3, you are correct - both nodes or manual intervention are
required if disk-tiebreaker isn't used...)

> By design, a two-node cluster can't be started with only
> one node, even with a quorum disk/partition that contributes votes.
> That was done to guard against split-brain.

You can start it with disk votes, too.  It currently doesn't work
correctly because of a bug in CMAN where it doesn't advertise qdisk
votes to the service manager.

Patch is here:

  http://people.redhat.com/lhh/cman-kernel-fixes.patch

(^^ Patrick made that patch, really...).

The requirement for fencing is *not* obviated by the use of qdisk,
however; CMAN can and will still fence the presumed-dead node(s) before
a quorum is established.

-- Lon