[Linux-cluster] Problem with gfs2 - waiting for i/o

[Steven Whitehouse]

Hi,

You are using a rather too old version of GFS2. I'd suggest either using
Fedora or the upstream kernel's until GFS2 appears in RHEL 5.1 as the
RHEL 5 version is rather out of date I'm afraid,

Steve.

On Wed, 2007-06-06 at 22:23 +0200, Jean-Marc jmc wrote:
> Hi list,
> I am new to cluster and gfs so my apologies if I missed something
> obvious.
> I am having problems using gfs2 filesystems where writing processes
> hang forever waiting for i/o. It does not seem to be locking (or even
> cluster) related since its reproducible on a local gfs2 filesystem
> with 'lock_nolock' as locking protocol.  
> 
> I can create and mount the gfs2 file system without any problems.
> Creating and writing to files works as well, but when I write larger
> amounts of data the process gets stuck waiting for i/o. It doesn't
> matter whether I amd using lvm or not thre results are the same. The
> hardware is working and I have no problem writing to an ext3
> filesystem on the same device. 
> 
> The problem can be reproduced by doing the following:
> 
> -------------------------------------------------------------------------------------------------
> [root at indigo ~]# mkfs.gfs2 -p lock_nolock /dev/cciss/c0d0p3 
> This will destroy any data on /dev/cciss/c0d0p3. 
>   It appears to contain a gfs2 filesystem.
> 
> Are you sure you want to proceed? [y/n] y
> 
> Device:                    /dev/cciss/c0d0p3
> Blocksize:                 4096
> Device Size                644.44 GB (168935523 blocks) 
> Filesystem Size:           644.44 GB (168935523 blocks)
> Journals:                  1
> Resource Groups:           2578
> Locking Protocol:          "lock_nolock"
> Lock Table:                "" 
> 
> [root at indigo ~]# mount -o locktable=xyz /dev/cciss/c0d0p3 /export
> 
> [root at indigo ~]# mount|grep /export 
> /dev/cciss/c0d0p3 on /export type gfs2
> (rw,locktable=xyz,localflocks,localcaching)
> 
> [root at indigo ~]# dd if=/dev/zero of=/export/8GB bs=1024k count=8000
> 
> ## A partial file is created, but the process never returns, it is
> forever waiting for i/o: 
> [root at indigo ~]# ls -l /export/;ps auxww|grep 8GB
> total 1995152
> -rw-r--r-- 1 root root 2039009280 Jun  1 07:12 8GB
> root      3857 42.8  0.0   4880  1584 pts/0    D    07:12   0:07 dd
> if /dev/zero of /export/8GB bs 1024k count 8000 
> ----------------------------------------------------------------------------------------------------
> 
> It's still possible to create new files to the file system but these
> processes are also waiting for i/o after a while.
> After the mounting there is nothing else appearing in dmesg or
> messages log. Since the processes are uninterruptible we are forced to
> reboot the machine in order to resolve the lockup. 
> 
> Basic info on the system setup:
> 
> HP dl380 G5 2 xeon cpu with dual cores each, 12 GB RAM
> local raid controller is p400 but we also reproduced the behaviour on
> the san disk (Emulex card)
> OS is RH 5 (32 bit), output of uname is: 
>     Linux indigo 2.6.18-8.1.4.el5xen #1 SMP Fri May 4 22:42:50 EDT
> 2007 i686 i686 i386 GNU/Linux
> 
> The same problem could also be reproduced on RH 4 and Ubuntu 7.04
> 
> Thanks in advance for your help!
> 
> /jmc
> 
> --
> Linux-cluster mailing list
> Linux-cluster at redhat.com
> https://www.redhat.com/mailman/listinfo/linux-cluster