[Linux-cluster] Kernel panic: GFS: Assertion failed on line 550 of file rgrp.c

[Kevin Anderson]

On Fri, 2006-02-10 at 00:17 +0700, Thai Duong wrote:
> Hi all,
> 
> My story is a little bit long so your patience is highly appreciated.
> I have a 4-node Oracle9iR2 RAC using GFS 6.0 as the cluster file
> system. Each node is running RHAS3 Update 6 for Itanium. We have 2
> main pools: oracle_u01 to store the $ORACLE_HOME, and oracle_u02 to
> store the datafile, controlfile, redolog...Many thanks to the
> stabilization of GFS, this production cluster have been running pretty
> well. Two days ago, the manager wants to have a backup cluster with
> the same configuration except it consits of only one node. Due to the
> lack of 64-bit servers, we got a node from the production cluster out
> to make it become the 1-node backup cluster. We make new LUNs in our
> EMC CX500 SAN, and clone the production's oracle_u01 and oracle_u02 to
> them. We are able to mount these pools as GFS file system onto the
> backup cluster's node. So far so good until we try to write to
> oracle_u02. Kernel panic: GFS: Assertion failed on line 550 of file
> rgrp.c. It's pretty weird cauze writing to oracle_u01 just works. They
> are just the same pool device containing GFS file system, arent they?
> We even can not run df against u02! We can mount and explore it using
> commands like cd, ls, dir...but not df! 

Purely speculation, but it sounds like when you snapped the LUN, there
was filesystem metadata that was not consistent on the storage.  This
can happen if you did not umount the filesystem or freeze it before
doing the clone.  Have you run fsck?

Kevin