[dm-devel] Re: [PATCH v3 0/4] dm-replicator: introduce new remote replication target

[James Bottomley]

On Wed, 2009-12-09 at 13:38 -0500, Mike Snitzer wrote:
> On Wed, Dec 09 2009 at  1:12pm -0500,
> James Bottomley <James.Bottomley at HansenPartnership.com> wrote:
> 
> > On Wed, 2009-12-09 at 05:39 -0500, Christoph Hellwig wrote:
> > > On Tue, Dec 08, 2009 at 12:42:27PM -0600, James Bottomley wrote:
> > > > Could I just echo Lars' statement.  With the upstream inclusion of drbd,
> > > > dm-replicator becomes a *third* replication system asking to be in
> > > > kernel.  It is definitely a kernel policy question of whether we want
> > > > three separate replicators, and so should be Cc'd to lkml so that people
> > > > interested in that can weigh in.
> > > 
> > > And unliley the previous two this one actually offers the benefit of
> > > beeing integrated with our major block device management framework.
> > 
> > md/nbd *is* integrated with a major block management framework.  The
> > fact that it's md not dm reflects the fact that it leverages the md
> > raid1 framework to perform the replication and merely uses nbd as a
> > remote block transmission pipe.  I'd submit this is the correct way to
> > do things.
> 
> Yes and no...
> 
> As someone who producticized md+nbd for a previous proprietary employer
> (standing on the shoulders of the work that was done by steeleye) I can
> say that md+nbd can provide the core plumbing but you need quite a bit
> of higher-level tools integration to make it _really_ approachable for
> the enterprise.  command line and UI interface and backend DB to store
> all relations, etc... And all sorts of nasty corner cases (e.g. split
> brain and double failure scenarios) are left as an exercise to the
> md+nbd user.

I didn't advocate using it as a framework ... I said it was done the
right way (leveraging the existing RAID engines).  What's actually
missing from it is the framework.

> > The problem now is that the md raid framework isn't integrated into dm,
> > but I think someone else is looking at that ...
> > 
> > > Interesting that the question comes up now after I was shot down for it
> > > in the drbd discussion.
> > 
> > So the value add of drbd over md/nbd is symmetric active.  I think that
> > could be pulled into the md raid infrastructure as well, but someone has
> > to figure out how.
> 
> md+nbd really isn't the way forward here IMHO..  if anything I think we
> need to focus on melding drbd and dm-replicator into the DM framework.

Actually, I think we need to focus on the goal, which should be a single
replication engine.  The problem with dm-replicator is that it brings
yet another network RAID-1 engine to the table.  The benefit is that it
does actually come with a framework.

The problem with network replicators is that they're hard to get right
and very time consuming to debug and test.  Since we've already invested
the correctness and testing effort twice over already (and it took
several years in each case) doing it yet again looks to be a big waste
to me.

> A single system management tool-chain and interface is increasingly
> important in the enterprise.

So isn't the way forwards then to prototype using this framework to
absorb both our existing in-kernel replicators?  A side bonus is that
the logging framework should be extensible to md to verify we're getting
it right.  If it's done correctly, it could even facilitate the eventual
raid unification goal of pulling together md and dm ... which would be a
huge benefit to the enterprise.

James