[dm-devel] dm-cache refusing to come up again after a crash

[Steinar H. Gunderson]

On Mon, Dec 09, 2013 at 10:28:11AM +0000, Joe Thornber wrote:
>> Well, first of all, it doesn't compile, since you use typename outside of
>> templates :-) Fixing that is easy, though. But afterwards:
> Grr, I thought that was fixed, what version of g++ are you using?

This is an Ubuntu 10.04 live CD, which was what I was having handy.
It works fine in a Debian wheezy live CD (which I switched to later).

>> So where do I want to go from there? cache_dump doesn't want to play with the
>> superblock because the checksum is bad... do I want cache_repair, then? Do I
>> want to take a backup of anything first?
> Ouch.  Could you go through what happened please?  Did dm-cache crash,
> or did the machine die for some other reason?

The machine hung. I don't know entirely why (I don't have the logs).
I rebooted, and it refused to take up the volume (this is what the original
post in this message is about). After booting to a live CD and running
cache_check and cache_dump, I was convinced there were no dirty blocks,
so I nuked the entire metadata volume (using dd from /dev/zero).

This made the machine boot again, but with tons of filesystem errors on
anything I'd written to in the last few months, so I restored from backup
(thankfully I do have working backups!). I also upgraded to 3.13-rc3 in the
hopes of fixing whatever issue in 3.12 originally caused this; however, as
reported in the other thread, this was notoriously unstable, and after the
third crash, I was back into the “won't boot, but cache_check says everything
is fine” mode.

That's the current status; it's now standing in a live CD and not doing much
useful. I miss my machine :-) (And I hope I haven't lost data again.) Will it
help if I upload a dump of the 512MB metadata volume somewhere?

/* Steinar */
-- 
Homepage: http://www.sesse.net/