[dm-devel] A simple, generic DM-API, for codecs/transcoders to use?

Navid Zamani navid.zamani at googlemail.com
Wed Mar 18 08:58:24 UTC 2009


Hello DM mailing list.

I a have a simple question:
Is there a way to use the device-mapper to hang an arbitrary program
in-between a real block device, and the virtual device created by the
device-mapper?
Like with LUKS. Only that I do not want to compress or encrypt. I want
to hang an error-correcting transcoder in between, so I can put ECC on
any block device I want.
The optimal thing would be a simple API, that is implemented in many
languages. (Like FUSE is available in many languages.)
Something like this would really be a powerful abstraction to have.

In case you are interested in the reasons: Modern consumer HDDs do not
have any ECC, but they get less and less reliable. It went so far, that
I saw random corruption without SMART telling me of any errors. (Of
course I eliminated the software as a cause.) RAID, backups and all that
stuff, does not help. Because this is a consumer case, and frankly, I do
not have the money for a RAID. Backups do not help, because you back up
your corrupt files with everything else, and only notice it, when it’s
too late.
So I decided to put my whole archive on a partititon with software-based
ECC. And because I could not find something that resembled this, I chose
to implement it myself.
I will program it in Haskell, and use cross-interleaved Reed-Solomon
codes (much like those on CDs) as a ECC. Is is a must-have to 1. fix
corruption on read (and re-write the fixed data), 2. report any tiny
problem to the user (log, mail, console message. all at once if
possible), and 3. protect the whole partition (else I would have used FUSE).

I bet there are more people out there who could use something like that.
Most certainly certain government agencies or scientists.
I wonder, what you have to say about this. Is there something like this
out there? Is there a better way? How would I implement it with the
device-mapper.

Navid




More information about the dm-devel mailing list