[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Linux-cluster] Shutdown/startup

David Teigland wrote:
On Thu, Mar 23, 2006 at 03:12:29PM -0600, Paul Tader wrote:

So how does one properly shutdown and/or start up a GFS 6.1 cluster. Documentation on the procedures are surprisingly sparse. For example, what I've experienced during shutdown are messages like:

CMAN: sendmsg failed: -101
CMAN: we are leaving the cluster.

from the node shutting down while the the other nodes report that this node needs to be fenced off. What?! Fenced during a controlled shutdown? Is that correct? During a complete cluster shutdown, of course quorum, at some point is lost, freezing the nodes that have not completely shutdown forcing a hard reset on those nodes. I'm probably missing something trivial here, but this doesn't seem right.

Start up also suffers from the "quorum" problem, but it seems to recovery once enough nodes are up.

You read the notes here about startup/shutdown?

To shutdown, you umount all fs's on all nodes, fence_tool leave on
all nodes, then leave the cluster on all.  To leave the cluster without
quorum getting in the way use 'cman_tool leave remove'.

To start up, have all nodes join the cluster with cman_tool join,
then have all nodes join the fence domain with fence_tool join,
then start other things.


Thanks Dave. The process detailed in this document brought the nodes down gracefully.

[root node]# umount /gfs_volume
[root node]# vgchange -aln
[root node]# killall clvmd
[root node]# fence_tool leave
[root node]# cman_tool leave -w
[root node]# killall ccsd
[root node]# (reboot/shutdown)

I'm going to incorporate this in to a revised shutdown init script and post what work.


Paul Tader  <ptader fnal gov> Computing Div/CSS Dept
Fermi National Accelerator Lab; PO Box 500 MS 369 Batavia, IL 60510-0500

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]