[Linux-cluster] Cluster among geographically separated nodes ?

[Gordan Bobic]

On Thu, 18 Jun 2009 14:51:32 +0200, Giuseppe Fuggiano
<giuseppe.fuggiano at gmail.com> wrote:
> 2009/6/18 Gordan Bobic <gordan at bobich.net>:
>> On Thu, 18 Jun 2009 17:11:42 +0530, Brahadambal Srinivasan
>> <brahadambal at gmail.com> wrote:
>>> Hi,
>>>
>>> I am trying to figure out if it is possible to create an RHCS cluster
>> among
>>> nodes that are in remote locations? If yes, then how are the following
>>> handled? :
>>>
>>> 1. Storage - how is the shared storage acheived?
>>
>> Same as it is achieved locally. It is up to your SAN to handle this in a
>> real-time, consistent way. You may want to look into DRBD
>> (http://www.drbd.org) for the block device level replication. Be aware,
>> however, that performance on the disk access front will be terrible,
>> because the latency will end up being limited by your ping time on the
>> WAN.
>> So instead of it having 0.1ms added via a local gigabit interconnect,
>> it'll
>> have 50-100ms added to it. Most applications will not produce usable
>> performance with this kind of disk I/O speed.
> 
> I am wondering if that will affect both read and write requests or
> only write/verify ones (which DRBD have to replicate using the
> network).

It'll affect both a lot of the time even if one site is passive/failover,
and pretty much all the time if it's an active-active configuration with
both sides handling load. DLM will end up bouncing and checking locks back
and forth between the sites. This will be the case with any real-time
distributed storage system that guarantees full consistency.

In other words, load sharing over a WAN will have unusable performance in
most cases. Within the same campus, it'd be OK, but between different
continents, I don't see it being viable. The real question is whether you
really need/want load sharing. If not, you can just use ext3 with DRBD in
active-passive mode with failover. Or you can use a more farm-like approach
where the servers are mostly serving data, and updates/writes can be
streamed from a single master using something like SeznamFS.

Gordan