[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[Linux-cluster] economy filesystem cluster


I'm planning to build a webserver cluster, and as part of that I'm looking for solutions that allows every node in the cluster to access the same filesystem. The most easy way would be via nfs, but my requirements state that there should be no single point of failure (ofcourse not completely possible but the cluster should not be affected by the downtime of 1 machine). A san or other some other piece of extra hardware is currently
not possible within the current budget.
The system will have a low number of writes (only some uploaded files and some generated templates but the majority of the load will be reads) but a rsync solution or something like that is not feasible since loadbalancing needs the file to be directly available on all

What I have:
- 1 loadbalancing machine
- 1 database server
- 2 webfrontends
- 1 management server (slave db / backup load balancer etc)
In the future I plan on adding some extra database servers + webfrontends

All machines are very similar and have (dual) xeon processors. The requirements are that all machines have access to the filesystem, and no single machine may affect the availability
of (a part of) the filesystem.
Searching the internet resulted in some possible solutions:
- GFS with only gnbd (http://gfs.wikidev.net/GNBD_installation).
 This only exports the specified partitions over the network and
has (in my mind) no advantages over using plain nfs (it adds no redundancy) - GFS with gnbd in combination with drbd (mentioned a few times on the mailing list). This looks promising but I couldn't find a definitive answer to the questions
 raised here on the mailinglist:
- drbd 0.7 only allows 1 node to have write-access. Is it possible to construct a simple failover scenario without serious risks of corruption when drbd has "failed-over"
   but gfs has not.
- drbd 0.8 seems to have support for multi(2)-master configuration, but is it stable
   enough for a production environment and can it work together with gfs
- GFS in combination with clvm (network raid?). Mentioned a few times here on the mailinglist but most posts claim it is not stable enough, and documentation
 seems completely missing.
- economy configuration from the GFS Administrator's Guide
 The problem with this is:
- is there a need to have separate gnbd servers? Or can the gnbd servers be run on
   the application servers.
- it is not documented how to configure this, and it is not clear whether this configuration
   gives me the redundancy I want.

What I was thinking of is the following:
- One node acts as a gnbd server
- Each node has his own disk
- Each node mounts a gnbd device.
- Each node creates a raid-1 (own disk + gnbd device)
- GFS is run on top of the raid-1

But it is not clear to me if this is feasible since I rely on a single gnbd
server. Maybe I can have 2 gnbd servers where the disks are synced
with drbd (0.8?), but that creates issues with fencing (according to some posts here).
And also the raid-1 should read only from it's local disk and only if that
fails it should read from the gnbd device, but I don't know if that is possible.
Or maybe clvm (network raid?) would be an option but I couldn't find
any documentation for that.

Can this be done with gfs / clvm / drbd or are there other solutions more
appropriate for this case? (other filesystems I've seen, like pvfs2/intermezzo/lustre,
are either not production ready, abandoned or don't have support for


Eric de Ruiter

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]