needs help, root inode gone after usb bus reset on sata disks

[Jelle de Jong]

Jelle de Jong wrote:
> Theodore Tso wrote:
>> On Tue, May 27, 2008 at 10:56:32AM +0200, Jelle de Jong wrote:
>>> I pluged in my USB to SATA converter in my harddisk that has an ext2
>>> filesystem. I mounted the partition, went to a directory that had a DVD
>>> image. I mounted the dvd image in the same directory and started
>>> watching the movie. After 40 minutes the movie stops.
>>
>> Were you doing anything else on the computer; where there any write
>> operations taking place?  If you were just reading from the
>> filesystem, the fact that your filesystem was that badly damaged makes
>> me deeply suspicious about your USB to SATA converter.
> 
> There was nothing else going on then watching a DVD form the disk. It 
> may have been an usb issue, but I am using the same sata usb converter 
> for several hours now without any problem. But an usb converter / power 
> failure should not be able to create so much damage when just reading 
> files...
> 
>>
>>> fsck.ext2 -p /dev/sdd1 did not work manual run is needed.
>>>
>>> On the first 500GB disk I did an fsck.ext2 -y /dev/sdd1 did did not
>>> fixed my disk it had still errors, I lost 35% of my data, but the
>>> partition was mountable again, and the files where in the lost+found
>>> directory.
>>
>> It looks like garbage was written into your block group descriptors,
>> but since the superblock looked OK, e2fsck -y tried its best, but in
>> this case it may have done more harm than good.  (In general, if you
>> see e2fsck asking permission to relocate an inode table; there's
>> something very wrong, and you probably want to say 'n' and do an image
>> level backup of the filesystem before proceeding.)
>>
>>> I don't want this to happen with the second 750GB harddisk, I would like
>>> all my data back.
>>
>> Well, there's no guarantee the same corruption will have taken place
>> on your other hard drive.  Running e2fsck -n on that second hard drive
>> and letting an expert examine it would be a good first step, *before*
>> blindly running e2fsck -y.
>  >
>> In the next version of e2fsprogs (in development in the git
>> repository), e2fsck will have the ability to create an "undo" log
>> which will make e2fsck -y safer, but personally I've always liked to
>> individually hit return to say 'yes' to each >question.
>>
>>> fsck.ext3 -n /dev/sdd1 > fsck-crash-info.txt 2>&1
>>> http://filebin.ca/mczmks/fsck-crash-info.zip
>>>
>>> What should I do? What commands do you want me to run to provide more
>>> info? How can i restore my root inode?
>>
>> So this is from your 500GB disk, as I understand it, right?  I'd
>> really need to see the results of "e2fsck -n" *before* you ran "e2fsck
>> -y" but seeing what I see there, taking an image-level backup before
>> you had begun would have been really good idea.
> 
> The log is of the second hard drive. I don't have a log of the first 
> hard drive, but it had very very similar outputs. Going to create an 
> image and hope an expert can tell me how to try fixing the file system.
> 
>> I'm not sure there's anythign you'll be able to do about restoring
>> your root inode.  But if it was just the root inode that was
>> destroyed, that's actually not a big deal; you'll just have files in
>> lost+found, and you can usually piece together the root directory
>> fairly easily.
>>
>> The bigger problem is the other parts of the filesystem that were
>> corrupted, due to what was apparently a hardware failure.  I'm
>> actually really not a fan of USB as an interconnect for disks, because
>> the cables can be flakey; it's not that hard for them to come lose,
>> which may have been what caused your USB<->SATA converter to flake
>> out, but it apparently did so in a very spectacular fashion.
> 
> The reason i used usb connections is power saving, just plug in the hard 
> drive you need. I think I will have an closer look at a placing my 
> harddrives in my server finding some way to hot-swap hot-powerplug 
> drivers enable and disable the power to harddrivers.
> 
>> When I have time I'll have to add a better automated hueristic to
>> e2fsck try to do this automatically (although even when I make e2fsck
>> -y smarter, there *still* will be cases where a human with experience
>> and intelligence and common sense will do better than a program), but
>> for now, if you see a message about wanting to relocate an inode
>> table, you'll want to look at the output of "dumpe2fs /dev/sdXX",
>> "dumpe2fs -o superblock=32768 /dev/sdXX", and "dumpe2fs -o
>> superblock=98304 /dev/sdXX" (these numbers are assuming a 4k
>> blocksize, which is the common default).  If the location of the inode
>> table blocks makes more sense when dumpe2fs is told to look at the
>> backup superblock at 32768, it may be that e2fsck -b 32768 /dev/sdXX
>> will do a better job of recovering the filesystem.
> 
> dumpe2fs /dev/sdXX
> dumpe2fs -o superblock=32768 /dev/sdXX
> dumpe2fs -o superblock=98304 /dev/sdXX
> e2fsck -b 32768 /dev/sdXX
> 
> Sound like a lot of experimentation, so I am going to make a backup first.
> 
> I do not have an journaling system on my disk, would it have been a lot 
> saver to have journaling on usb disk? and what about an auto sync option 
> flag for usb disks?

So, it took 14 hours to pump over the 750G to an other disk, but i hope 
it went ok. I executed the below command created the logs and hope 
somebody can tell me what to do next?

dd if=/dev/sdb of=/dev/sda > dd-run-info.txt 2>&1
dumpe2fs /dev/sda1 > dumpe2fs-info-sda1.txt 2>&1
dumpe2fs -ob 32768 /dev/sda1 > dumpe2fs-32768-info-sda1.txt 2>&1
dumpe2fs -ob 98304 /dev/sda1 > dumpe2fs-98304-info-sda1.txt 2>&1
e2fsck -b 32768 /dev/sda1 (need terminal for interactive repairs)

http://www.powercraft.nl/temp/fsck-crash-info.txt.gz
http://www.powercraft.nl/temp/dumpe2fs-info-sda1.txt.gz
http://www.powercraft.nl/temp/dumpe2fs-32768-info-sda1.txt.gz
http://www.powercraft.nl/temp/dumpe2fs-98304-info-sda1.txt.gz

Thanks in advance,

Jelle