[Linux-cluster] Re: [cgl_discussion] Re: [dcl_discussion] Cluster summit materials

[Daniel Phillips]

On Wednesday 11 August 2004 17:24, Daniel McNeil wrote:
> > IMHO, for the time being only failure detection and failover really
> > has to be unified, and that is CMAN, including interaction with
> > other bits and pieces, i.e., Magma and fencing, and hopefully other
> > systems like Lars' SCRAT.  As far as CMAN goes, Lars and Alan seem
> > to be the main parties outside Red Hat.  Lon and Patrick are most
> > active inside Red Hat.  I think we'd advance fastest if they start
> > hacking each other's code (anybody I just overlooked, please
> > bellow).
>
> I not sure what you mean by "failure detection and failover".
> Do you mean node failure detection and consensus membership change?

I mean anything in the cluster that can fail and be reinstantiated.  
This would include server processes for cluster block devices such as 
the ones I've designed, as well as whole nodes.  It would also include 
communication paths, such as socket connections.  But by now you may 
have detected a bias against trying to deal with the latter in a 
one-size-fits-all automagic, never-stop-never-give-up cluster 
communications thingamajig layer.  What we really need is just a 
framework for failure detection, including methods supplied by various 
cluster components, and methods for re-instantiating failed components.

Note note note: while a "cluster component" could conceivably be a whole 
node, that's a special case and we really need to cater to the case 
that will eventually be much more common, where cluster nodes may be 
doing all kinds of other things besides just participating in clusters.  
So by "cluster component" I really mean something closer to "task".

> I thought Magma is just redhat's backward compatibility layer.
> What "interaction" are you worried about?

You might want to ask Lon about that...

> How fencing integrates and when it occurs might be issues we
> will need to think about more.

Understatement of the day.

> How can the DLM go to Andrew without a membership layer to
> provide membership?

By having a simple registration api that allows one to register a 
membership layer, in place of what is there now, i.e., function links 
between modules.

> > > So you can call the core service "membership", but what we really
> > > need is membership/communication, which is what cman provides. 
> > > Do you have another suggestion for this?  TIPC + membership?
> >
> > I think you really mean "connection manager", not "communication
> > service"  I'll step back from this now and watch you guys sort it
> > out :-)
>
> I think John really does mean communication.  For high availability,
> the cluster should have no single point of failure.  This usually
> means multiple ethernet links.

But it's not the business of the cluster framework to operate the links, 
only to know when they have failed and to be able to arrange for new 
connections.  So John really does mean "connection" and not 
"communication", I hope.

> (I assume CMAN supports multiple 
> links).  To determine membership there needs to be a way of sending
> messages between the nodes to determine membership.  Ideally, losing
> one ethernet link could/would be handle without causing any
> membership change.

"Ideally" is not a strong enough word, imho.

> This kind of intra-cluster communication would be valuable for
> other cluster components as well.  Example: a cluster snapshot :)
> or cluster mirror device should be able to send messages to
> other nodes in the cluster without having to worry about which
> specific link to use and what to do if a link fails.  This would
> also be valuable for the DLM.

OK, we've seen lots of warnings about not getting derailed by trying to 
invent the perfect cluster communication system, we should heed those 
warnings.  Instead, let's get down to precise specification of the 
methods we need to have, and compare it to what already exists, for 
establishing and re-establishing connections.

> Does CMAN provide this kind of functionality?  If so, then it
> really is a communication service.

http://people.redhat.com/~teigland/sca.pdf

Regards,

Daniel