[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Linux-cluster] qdiskd: Updated votes configuration not used even after restart



Mrugesh Karnik wrote:
Hi,

I have a three node cluster, of which, one node is perpetually offline. I have the cluster configured with qdisk. Since the third node is always offline, I configured the quorum settings for a two node cluster:

* 1 vote assigned to each node
* 1 vote assigned to qdisk
* expected votes set to 3

The configuration worked perfectly well.

I then needed to add a fourth node to the cluster, so I updated the configuration as follows:

* 1 vote assigned to each node
* 3 votes assigned to qdisk
* expected votes set to 4

I then did a ccs_tool update.

To have the value of qdisk votes updated, I then did the following:

* cman_tool expected -e 3

so as not to lose quorum while the qdisk was offline.

I restarted qdiskd on each node in turn, yet the value of votes was not updated.

I then stopped qdiskd on _all_ nodes and then started it up on each node one-
by-one. Yet, again, the value of votes assigned to the qdisk was not updated.

Here's the debug output from when qdiskd starts up:
May 19 13:36:52 eru qdiskd[18691]: <debug> 0 heuristics loaded
May 19 13:36:52 eru qdiskd[18691]: <debug> Quorum Daemon: 0 heuristics, 5 interval, 5 tko, 3 votes
May 19 13:36:52 eru qdiskd[18691]: <debug> Run Flags: 00000031
May 19 13:36:52 eru qdiskd[18692]: <info> Quorum Daemon Initializing
May 19 13:36:52 eru qdiskd[18692]: <debug> I/O Size: 512  Page Size: 4096
May 19 13:36:52 eru qdiskd[18692]: <debug> Permanently setting score to 1/1
May 19 13:37:02 eru qdiskd[18692]: <debug> Node 2 is UP
May 19 13:37:13 eru qdiskd[18692]: <debug> Node 4 is UP
May 19 13:37:18 eru qdiskd[18692]: <info> Initial score 1/1
May 19 13:37:18 eru qdiskd[18692]: <info> Initialization complete
May 19 13:37:18 eru qdiskd[18692]: <notice> Score sufficient for master operation (1/1; required=1); upgrading
May 19 13:37:28 eru qdiskd[18692]: <debug> Making bid for master
May 19 13:37:38 eru qdiskd[18692]: <info> Assuming master role

Here's the output of cman_tool status, on the same node:
Version: 6.1.0
Config Version: 35
Cluster Name: dom0cluster
Cluster Id: 10036
Cluster Member: Yes
Cluster Generation: 784
Membership state: Cluster-Member
Nodes: 3
Expected votes: 4
Quorum device votes: 1
Total votes: 4
Quorum: 3
Active subsystems: 11
Flags: Dirty
Ports Bound: 0 11 177
Node name: eru.san.arda.geodesic.net
Node ID: 1
Multicast addresses: 229.192.0.1
Node addresses: 192.168.10.2

The same problem is seen on all the nodes. The updated configuration is read correctly by qdiskd, but once started, the updated votes value isn't used.


So, once registered, cman doesn't accept new registrations or changes to the registration - meaning that votes can't currently be changed. This obviously would explain both behaviors you saw.

It looks like we could safely add a block to do_cmd_register_quorum_dev() function that does something like:

 - if a quorum device exists and it is being reregistered with the same
   name, just change the votes and recalculate quorum

-- Lon


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]