[linux-lvm] Data deduplication in LVM?

[Ray Van Dolson]

On Wed, Jun 10, 2009 at 12:30:25PM -0700, Les Mikesell wrote:
> Roy Sigurd Karlsbakk wrote:
> >
> >> Roy Sigurd Karlsbakk wrote:
> >>> Hi all
> >>> I've been reading up a little about data deduplication, and have been 
> >>> in search for an OSS filesystem with dedup without much luck. While 
> >>> testing snapshots and so on in LVM, I started wondering if dedup 
> >>> would be better off in LVM than in the filesystem. Would it be 
> >>> possible/efficient to add dedup to the LVM layer, or perhaps a layer 
> >>> above LVM? This could make dedup work for all or most of filesystems. 
> >>> Make a hash table with 4k (or whatever) blocks, make virtual blocks 
> >>> pointing to the physical blocks and run a remapping/deduping job at 
> >>> night. If written to, copy-on-write could be used to increase speed.
> >>> Is this nonsense, or might it be an idea?
> >>
> >> This is "supposed" to be coming in the next OpenSolaris/ZFS release 
> >> (per the roadmap with the just-released 2009.06 version).
> > 
> > 
> > What about Linux/LVM? Or did I misunderstand you?
> 
> I thought the question was about OSS...  I wouldn't hold my breath 
> waiting for a Linux/LVM version - and for that matter I'll believe the 
> ZFS release when I see it, but at least it is being planned and could be 
> less than a year away.
> 

Also, both btrfs and tux3 are planning on adding dedup support.

Ray