[Linux-cluster] qdiskd: Updated votes configuration not used even after restart

[Mrugesh Karnik]

Hi,

I have a three node cluster, of which, one node is perpetually offline. I have 
the cluster configured with qdisk. Since the third node is always offline, I 
configured the quorum settings for a two node cluster:

* 1 vote assigned to each node
* 1 vote assigned to qdisk
* expected votes set to 3

The configuration worked perfectly well.

I then needed to add a fourth node to the cluster, so I updated the 
configuration as follows:

* 1 vote assigned to each node
* 3 votes assigned to qdisk
* expected votes set to 4

I then did a ccs_tool update.

To have the value of qdisk votes updated, I then did the following:

* cman_tool expected -e 3

so as not to lose quorum while the qdisk was offline.

I restarted qdiskd on each node in turn, yet the value of votes was not 
updated.

I then stopped qdiskd on _all_ nodes and then started it up on each node one-
by-one. Yet, again, the value of votes assigned to the qdisk was not updated.

Here's the debug output from when qdiskd starts up:
May 19 13:36:52 eru qdiskd[18691]: <debug> 0 heuristics loaded
May 19 13:36:52 eru qdiskd[18691]: <debug> Quorum Daemon: 0 heuristics, 5 
interval, 5 tko, 3 votes
May 19 13:36:52 eru qdiskd[18691]: <debug> Run Flags: 00000031
May 19 13:36:52 eru qdiskd[18692]: <info> Quorum Daemon Initializing
May 19 13:36:52 eru qdiskd[18692]: <debug> I/O Size: 512  Page Size: 4096
May 19 13:36:52 eru qdiskd[18692]: <debug> Permanently setting score to 1/1
May 19 13:37:02 eru qdiskd[18692]: <debug> Node 2 is UP
May 19 13:37:13 eru qdiskd[18692]: <debug> Node 4 is UP
May 19 13:37:18 eru qdiskd[18692]: <info> Initial score 1/1
May 19 13:37:18 eru qdiskd[18692]: <info> Initialization complete
May 19 13:37:18 eru qdiskd[18692]: <notice> Score sufficient for master 
operation (1/1; required=1); upgrading
May 19 13:37:28 eru qdiskd[18692]: <debug> Making bid for master
May 19 13:37:38 eru qdiskd[18692]: <info> Assuming master role

Here's the output of cman_tool status, on the same node:
Version: 6.1.0
Config Version: 35
Cluster Name: dom0cluster
Cluster Id: 10036
Cluster Member: Yes
Cluster Generation: 784
Membership state: Cluster-Member
Nodes: 3
Expected votes: 4
Quorum device votes: 1
Total votes: 4
Quorum: 3
Active subsystems: 11
Flags: Dirty
Ports Bound: 0 11 177
Node name: eru.san.arda.geodesic.net
Node ID: 1
Multicast addresses: 229.192.0.1
Node addresses: 192.168.10.2

The same problem is seen on all the nodes. The updated configuration is read 
correctly by qdiskd, but once started, the updated votes value isn't used.

Thanks,
Mrugesh