Freezing GFS (Re: [Linux-cluster] GFS and CLVM-snapshot)

[Kenji Wakamiya]

Hello,
I'm sorry for a long interval.

AJ Lewis wrote:
> On Thu, May 12, 2005 at 11:28:37AM +0900, Kenji Wakamiya wrote:
>>By the way, sometimes I get warnings from gfs_fsck for snapshot-LV,
>>even though I froze GFS before doing lvcreate -s.
>>For example, the followings are.  Is this usual thing?
>>
>>  # gfs_fsck -y /dev/vg0/lv0ss0
>>  Initializing fsck
>>  Starting pass1
>>  Pass1 complete
>>  Starting pass1b
>>  Pass1b complete
>>  Starting pass1c
>>  Pass1c complete
>>  Starting pass2
>>  Pass2 complete
>>  Starting pass3
>>  Pass3 complete
>>  Starting pass4
>>  Found unlinked inode at 1929233
>>	  Adjusting freemeta block count (59 -> 60).
>>	  Adjusting used dinode block count (10 -> 9).
>>  l+f directory at 29
>>  Added inode #1929233 to l+f dir
>>  Found unlinked inode at 1800635
>>  Added inode #1800635 to l+f dir
>>  Link count inconsistent for inode 25 - 5 6
>>  Link count updated for inode 25
>>  Pass4 complete
>>  Starting pass5
>>  ondisk and fsck bitmaps differ at block 29
>>  Succeeded.
>>  used inode count inconsistent: is 9 should be 10
>>  free meta count inconsistent: is 60 should be 59
>>  Pass5 complete
>>  #
> 
> 
> Interesting.  Have you noticed what inode gets stuck in l+f?

I'm using IlohaMail-0.9-20050415, a PHP based POP/IMAP web mailer
- http://blog.ilohamail.org/ on 3-nodes GFS cluster with a server
load balancer.  The lost file is produced in this way:

1. Reload all three HTML frames or only folder-tree frame of IMAP
   client mode of IlohaMail, and wait.
2. Just after reload is complete, execute "gfs_tool freeze" on
   any node of GFS cluster (I tried also on all nodes).
3. Create a block/volume level snapshot on file server.  I tried
   GNBD/LVM/LinuxBox and iSCSI(sfnet-initiator-4.0.2.1)/NetApp.
4. Execute gfs_tool unfreeze.
5. Import the snapshot LV/LUN on any Linux machine, and execute
   gfs_fsck.

By this procedure, the lost file is always an IlohaMail's cache
file for IMAP folder list and each property.
(/mnt/gfs/IlohaMail/data/cache/$USER.$IMAPSERVERNAME/folders)
The inconsistency is sometimes not caused.

I also tried to sync before/after freezing GFS, but nothing seemed
to make difference.

Freezing GFS suspends filesystem's activity, but doesn't it
ensure filesystem level consisntency?

Right now I'm using GFS on FC4-test3 with the newest kernel RPMs
and the newest Cluster/GFS RPMs in:
http://download.fedora.redhat.com/pub/fedora/linux/core/development/i386/Fedora/RPMS/

Thanks,

-- Kenji