[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Linux-cluster] GFS join hang



Unfortunately, in  a rush of desperation I rebooted the complete cluster and am now waiting for a new gfs_fsck to finish.
However I did find there was a segfault in one (unused) ethernet interface. Maybe this causes communication problems (lost ack...). 
I will test this hypothesis tomorrow morning (I am on my way out of office because of the time zone...).

  Thanks for your very useful hint and explanation.

David Teigland wrote:
On Thu, Apr 20, 2006 at 09:56:20AM +0200, Fernando Nino wrote:
  
I am running GFS 6.1 with dlm on a cluster (4 nodes + front-end) of 
dual-headed Opterons and RHEL4U3. Because of some problems (kernel 
panic...) I had to hard boot some nodes of the cluster.  Now, some gfs 
partitions won't mount.  They will simply keep waiting forever for the 
"join" of the GFS group:

So... three questions:

- What is the join exactly doing ? Cluster status is fine, everybody is 
member ...
    

>From all 5 nodes it would be good to see:
- cman_tool services
- /var/log/messages
- /proc/cluster/lock_dlm/debug

  
- What does the status code mean in the cman_tool output ?
S-2,2,4
    

S-2: join event state is SEST_JOIN_ACKWAIT
,2: join event flag is SEFL_ALLOW_JOIN
,4: number of acks to our join request is 4

So, the node is waiting for acks to its join request.  It needs 5 but has
only got 4, someone hasn't sent a reply for some reason.  We might be able
to figure out who and why given all the info from the other nodes.
Rebooting the node that's not replied might resolve things.

Dave

  

--

Fernando NIÑO CNES - BPi 2102
Medias-France/IRD 18, Av. Edouard Belin
Tél: 05.61.27.40.74 31401 Toulouse Cedex 9

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]