input/output errors

[Rick Stevens]

Benjamin Hornberger wrote:
> Hi all, 
> 
> we have a machine (RHEL AS 3) on which last night suddenly a lot of input /
> output errors occured. Basically, two partitions are unusable. 
> 
> Both of these partitions are RAID 1 devices which share the same two IDE
> hard disks (/dev/hda and /dev/hdc are two 250 GB drives, and /dev/md0 and
> /dev/md1 are two RAID 1 devices which take 70 and 180 GB from each drive,
> respectively). 
> 
> Any hints? I looked into fsck, but I am not sure what is the right thing to
> do. 

Ben,

fsck is a tool that will (hopefully) fix filesystem inconsistencies.
You should boot up in single user mode and run fsck against the two
filesystems that have issues.  Note that you may lose some data when
you do that.  Data that can't be reattached to their files will end up
in the "lost+found" directory of the filesystem being fsck'd and given
filenames that refer to their inode number.  You may be able to rebuild
the file by looking at those files, but it's a tedious, error-fraught
process.

Since you set up RAID 1, you should first split the RAIDs into two disks
and see if either disk has clean versions of the data.  If so, you may
be able to purge the bad drive and recreate the RAID.

The most important thing to figure out is why you started getting I/O
errors in the first place.  Is one of the drives dying?  Did you have
a power glitch?  Did a RAM stick start acting weird?  You must fix the
underlying issue or you're just going to get a repeat of this event.
----------------------------------------------------------------------
- Rick Stevens, Senior Systems Engineer     rstevens at vitalstream.com -
- VitalStream, Inc.                       http://www.vitalstream.com -
-                                                                    -
-        Brain:  The organ with which we think that we think.        -
----------------------------------------------------------------------