[Linux-cluster] 4U5 CSS/CMAN/fence quorum confusion
Robert Clark
cluster at defuturo.co.uk
Sun Jun 10 19:53:45 UTC 2007
On Fri, 2007-06-08 at 16:20 +0100, Robert Clark wrote:
> On Fri, 2007-06-08 at 13:04 +0100, Patrick Caulfield wrote:
> > Robert Clark wrote:
> > > Does anyone know what might cause ccsd to continue to refuse
> > > connections for a lack of quorum after cman has decided the cluster is
> > > quorate?
>
> > The usual cause of this is the magma plugins either not being installed in the
> > right place or even at all. "magma_tool list" will show you which plugins are
> > installed, for CMAN you need the magma_sm.so plugin.
>
> Thanks for the quick reply. I put "magma_tool list" into the script
> just before and after trying to start fenced. The output both times is:
>
> Magma: Checking plugins in /lib/magma
>
> File Status Message
> ---- ------ -------
> magma_gulm.so [OK] GuLM Plugin v1.0.5
> magma_sm.so [OK] CMAN/SM Plugin v1.1.7.4
>
> Magma: 2 plugins available
>
> When I added "magma_tool quorum" as well, it reported "Connect
> failure: No cluster running?".
I've managed to get an strace of ccsd during the boot and it turned up
some interesting lines, which I've interspersed with selected log
entries:
Jun 8 22:20:27 localhost ccsd[2981]: Starting ccsd 1.0.10:
Jun 8 22:20:27 localhost kernel: CMAN 2.6.9-50.2 (built May 31 2007 15:39:24) installed
Jun 8 22:20:27 localhost kernel: NET: Registered protocol family 30
Jun 8 22:20:27 localhost ccsd[2981]: Built: May 31 2007 15:48:09
Jun 8 22:20:27 localhost ccsd[2981]: Copyright (C) Red Hat, Inc. 2004 All rights reserved.
Jun 8 22:20:27 localhost kernel: DLM 2.6.9-46.16 (built May 31 2007 15:45:51) installed
Jun 8 22:20:28 localhost ccsd[2981]: cluster.conf (cluster name = defuturo_test, version = 2) found.
Jun 8 22:20:28 localhost kernel: CMAN: Waiting to join or form a Linux-cluster
2990 22:20:28 stat64("/dev/dlm-control", 0xf6f2c104) = -1 ENOENT (No such file or directory)
Jun 8 22:20:29 localhost kernel: CMAN: sending membership request
2990 22:20:29 stat64("/dev/dlm-control", 0xf6f2c104) = -1 ENOENT (No such file or directory)
Jun 8 22:20:30 localhost kernel: CMAN: got node tamarillo
Jun 8 22:20:30 localhost kernel: CMAN: got node guava
Jun 8 22:20:30 localhost kernel: CMAN: quorum regained, resuming activity
Jun 8 22:20:30 localhost cman: startup succeeded
2990 22:20:30 stat64("/dev/dlm-control", 0xf6f2c104) = -1 ENOENT (No such file or directory)
2990 22:20:31 stat64("/dev/dlm-control", 0xf6f2c104) = -1 ENOENT (No such file or directory)
2990 22:20:32 stat64("/dev/dlm-control", 0xf6f2c104) = -1 ENOENT (No such file or directory)
2990 22:20:33 stat64("/dev/dlm-control", 0xf6f2c104) = -1 ENOENT (No such file or directory)
2990 22:20:34 stat64("/dev/dlm-control", 0xf6f2c104) = -1 ENOENT (No such file or directory)
2990 22:20:35 stat64("/dev/dlm-control", 0xf6f2c104) = -1 ENOENT (No such file or directory)
2990 22:20:36 stat64("/dev/dlm-control", 0xf6f2c104) = -1 ENOENT (No such file or directory)
2990 22:20:37 stat64("/dev/dlm-control", 0xf6f2c104) = -1 ENOENT (No such file or directory)
2990 22:20:38 stat64("/dev/dlm-control", 0xf6f2c104) = -1 ENOENT (No such file or directory)
2990 22:20:39 stat64("/dev/dlm-control", 0xf6f2c104) = -1 ENOENT (No such file or directory)
2990 22:20:40 stat64("/dev/dlm-control", 0xf6f2c104) = -1 ENOENT (No such file or directory)
2990 22:20:41 stat64("/dev/dlm-control", 0xf6f2c104) = -1 ENOENT (No such file or directory)
2990 22:20:42 stat64("/dev/dlm-control", {st_mode=S_IFCHR|0600, st_rdev=makedev(10, 62), ...}) = 0
Jun 8 22:20:42 kiwano ccsd[2981]: Connected to cluster infrastruture via: CMAN/SM Plugin v1.1.7.4
Jun 8 22:20:42 kiwano ccsd[2981]: Initial status:: Quorate
So, it looks like the problem is that the appearance of /dev/dlm-control
is being delayed in the 4U5 cluster.
Robert
More information about the Linux-cluster
mailing list