[Linux-cluster] Is this even possible with GFS, replacing NFS

[Brandon Lamb]

I am on about day 4 or so of researching filesystems and cluster
software, and ready to pull my hair out.

First let me describe my situation.

I have a 20 disk scsi raid-10. The server that it is connected to has
4 gigs of ram, and exports a single directory via NFS. We only have a
150gig partition, using about 100 of it. We use this for our mail data
store, 2.8 million files using maildir format.

We have 5 linux boxes connected via NFS over gigabit (NOT using jumbo
frames) and they all write to the users maildirs via this nfs mount.

Now, this IS working and has been for over 1.5-2 years, however it is
slow as hell.

Here is where my question comes in. We are planning on splitting the
current raid into two 10 disk raid10s so that we can replicate and
have a live spare. That part I already am 90% sure of how I will
handle. But in rebuilding our mail system I began looking at NFS
alternatives and came down to OpenAFS and RedHat GFS.

After a ton of reading and a few usenet posts it came down to AFS
probably was not the solution and to try GFS.

Ok, so my current plan is figuring out how to mount the raid10
parition /dev/sda1 on the "server" using iSCSI on the 5 mail heads. I
got the iSCSI part working in a test setup with 2 machines, 1 being
the sever and 1 being the client. This worked great, doing a test
transfer of a 771meg maildir took 24 secs compared to the same test
over NFS of 6 mins 32 secs.

SO. Now I am at the point that I THINK I should be able to setup iscsi
on 5 mail heads to mount this server's /dev/sda1 over gigabit ethernet
using jumbo frames on a private switch (so bandwidth should not be a
problem?) and then setup GFS as the filesystem for this /dev/sda1 so
that all 5 can mount it RW.

So is this actually possible or am I wasting my time? At this point I
FINALLY got the damn cluster-1.0.3 from cvs to compile with the latest
2.6.19.2 kernel and some hack files from kernel.org for fs/dlm/
Kconfig, lowcomms-tcp, locomms-sctp, Makefile and got GFS to compile
as kernel modules and the cluster stuff compiled.

But I cant seem to get it to work, cluster.conf is complaining about
stuff and whatnot, hopefully more reading will give me answers, but is
all this work and research in vane?

My current test setup is using FC6, kernel 2.6.19.2, i have the
cluster-1.0.3, i installed gfs2-utils, i have the iscsi target. From
what I can tell I just need to finish getting GFS to work and then i
should be able to mount my first gfs filesystem over iscsi on my test
client machine.

Sorry for the long post, hopefully I didnt forget anything.