[Linux-cluster] lost quorum, but the cluster services and GFS are still up

Lon Hohberger lhh at redhat.com
Mon Mar 13 18:35:59 UTC 2006


On Mon, 2006-03-13 at 10:31 -0700, Dex Chen wrote:
> Hi,
> 
>  
> 
> I believe that I saw something unusual here.
> 
>  
> 
> I have a 3 node cluster (with GFS) using CMAN. After I shutdown 2
> nodes in short time span, the cluster shows it lost quorum, but I run
> the clustat on the third node, and clustat shows the cluster has 3
> nodes (2 are offline) and the other services are up. I was able to
> access/read the share storage. CMAN_TOOL shows cluster lost quorum and
> the activity is blocked. What I expected is that I should not allow
> accessing the shared storage and other services at all when the
> cluster lost the quorum. Anyone has seen the similar things?
> What/where should I look into?

CMAN is supposed to deliver (more or less) a STATECHANGE event to
clients.  At that point, quorum is checked by rgmanager, and if the
cluster is no longer quorate, it halts all services immediately.

Are there anything in the logs which would indicate this?  It would look
like:

   <emerg> #1 Quorum Dissolved

Given that you can still access service data (e.g. clustat reports
something), that means that rgmanager can still acquire locks for some
reason (it takes DLM locks before giving out service data...).

Does clustat report that the cluster is quorate or not?

-- Lon




More information about the Linux-cluster mailing list