[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[Linux-cluster] managing GFS corruption on large FS

hi all

We have a large GFS consisting of 4 TB of maildir data. We have a corruption on this GFS which causes nodes to be withdrawn intermittently.

The cause of the fs corruption is due to user error and lack of documentation (initially not having the clustered flag enabled on the VG when growing the LV/GFS). We now know better, and will avoid this particular cause of corruption. However, management wants to know from us how we can prevent corruption, or minimize the downtime incurred if this should happen again.

For the current problem, since a gfs_fsck will take too long (we cannot afford the 1 - 3 days of downtime it will take to complete the fsck), we are planning to migrate the data to a new GFS, and at the same time set up the new environment optimally to cause the minimum of downtime, if a corruption were to happen again.

One option is to split the one big GFS into a number of smaller GFS's. Unfortunately, our environment does not lean itself to being split up in (for example) a number of 200GB GFS's. Also, this negates a lot of the advantages of GFS (e.g. having your storage consolidated onto one big GFS, and scaling it out by growing the GFS and adding nodes).

I would really like to know how others on this list manage the threat/risk of FS corruption, and the corruption itself, if it does happen. Also, w.r.t. data protection, if you do snapshots, SAN-based mirroring, backup to disk/tape, I would really appreciate it if you could give me detail information like
a) mechanism (e.g snaps, backup, etc)
b) type of data (e.g. many small files)
c) size of GFS
d) the time it takes to perform the action

thank you
fn:Riaan van Niekerk
n:van Niekerk;Riaan
org:Obsidian Systems;Obsidian Red Hat Consulting
email;internet:riaan obsidian co za
title:Systems Architect
tel;work:+27 11 792 6500
tel;fax:+27 11 792 6522
tel;cell:+27 82 921 8768

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]