[linux-lvm] Drive gone bad, now what?

[Chris Cox]

(orginally wasn't going to send this.. but changed my mind)
First, let me say that 100's of GB is nothing today.
A good LTO2 unit will do the trick quite nicely.  The problem
is that we are talking about 100's of TB now.  Even if
you have a 100 tape LTO2 drive or multiple LTO2 arrays
capable of moving 50M/sec+... you still end up in a crunch.

SAN folks will tell you to live mirror to a remote DR site...
but the pipe between sites could be VERY expensive (if you
have the money, it is definitely something worth considering
though).  What I'm suggesting below.. and I've tried to
use LVM... is the use of mirrored drives and the idea of
archival of hard disk storage.

Gert van der Knokke wrote:
...
> Now we have to find a reliable resizable/expandable filesystem (or 
> resizable/expandable block device system) on this hardware.

Mirrored drives.  So a logical drive includes a mirror as well.
You'll need a raid controller that can configure many drive instances
(drive + mirror).  Use LVM to add new logical drives into your
volume group, then extend your volumes and resize your filesystems.

(ascii art)

   pv1       pv2
+-----+   +-----+
| d 1 |   | d 2 |
+-----+   +-----+
| m 1 |   | m 2 |
+-----+   +-----+

pvcreate on pv1 and pv2 (can add pv3, etc later as needed)
put pv1, pv2 into a vg (extend with additional pv's later)
(if you like the pv's could be a set of mirrored drives
which are also striped).

Mirrored set could be pulled and archived at a storage center or
moved to a disaster recovery site.  Of course, this does imply
the ability to take out the entire range of mirrors safely.
You'll have to look at your HW raid controller to see what
is possible.  Afterwards, replace the "m" drives and let them
rebuild.

You may ask about joining the drives at the HW RAID level.. but as
mentioned, this might not give you the flexibility desired (see
the idea of striped sets mentioned earlier).

There are probably better solutions to this... obviously some
high end SAN devices may be better suited... not sure about
Linux filesystem compatibility though).

Just an idea.

Today, we're using a 5 TB disk based cache feeding into a
72 tape LTO library with 2 drives... adequate for our needs,
but we just don't have the storage needs that some require.
Moving forward, the idea of using off site disk storage,
off site DR replication or live DR mirroring are things we'll
haved to look at.