[Linux-cluster] RHEL5 GFS2 - 2 node - node fenced when writing

[David Teigland]

On Wed, Jun 06, 2007 at 07:27:42PM -0400, nrbwpi at gmail.com wrote:
> Hello,
> 
> Installed RHEL5 on a new two node cluster with Shared FC storage.  The two
> shared storage boxes are each split into 6.9TB LUNs for a total of 4 -
> 6.9TBLUNS.  Each machine is connected via a single 100Mb connection to
> a switch
> and a single FC connection to a FC switch.
> 
> The 4 LUNs have LVM on them with GFS2.  The file systems are mountable from
> each box.  When performing a script dd write of zeros in 250MB file sizes to
> the file system from each box to different LUNS, one of the nodes in the
> cluster is fenced by the other one.  File size does not seem to matter.
> 
> My first guess at the problem was the heartbeat timeout in openais.  In the
> cluster.conf below I added the totem line to hopefully raise the timeout to
> 10 seconds.  This however did not resolve the problem.  Both boxes are
> running the latest updates as of 2 days ago from up2date.
> 
> Below is the cluster.conf and what is seen in the logs.  Any suggestions
> would be greatly appreciated.

Did you see whether the fenced node panicked or hung before it was
rebooted?  I'd try the same thing with GFS1 (everything stays the same,
just 'mkfs -t gfs ...' and 'mount -t gfs ...', and make sure you have the
gfs kernel module loaded.)

Dave