[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Linux-cluster] iSCSI GFS

On Mon, 28 Jan 2008, isplist logicore net wrote:

How much I/O do you actually need?

My concern is not so much what I need up front but the growth path should I
start needing to add a lot of storage. LAMP services can sometimes grow very
quickly, immediately need endless ongoing storage space for uploaded media and
playback, not to mention the web services themselves.

Yes, but we are talking 10s od Gb here before you may need to alter your approach.

Like I mentioned, I've seen not being prepared for growth and it's not fun. I
would hate to have to keep changing out technologies once things get going
because I didn't choose a nice flexible solution up front.

creates virtual software RAID stripe over them. It then exports this
back out via iSCSI. All the client nodes then connect to the single big
iSCSI node that is the aggregator.

Do you know of any software which helps to keep track of all this? This is an
interesting idea. I think I understand it and want to give it a try. I have
various types of storage where this would be a good solution.

It's pretty simple to set up. You just need to be familliar with iSCSI tools and software RAID tools, all of which are almost certainly in your distro's apt/yum repositories.

Let's see if I've got this right.

I need a machine which will become the aggregator, plenty of memory,
multi-port Ethernet card and of course an FC HBA.
FC storage will be attached to this machine. Then, iSCSI storage targets will
also export to this machine.

Not quite sure I follow this - you want to use FC storage and combine it with iSCSI storage into a bigger iSCSI storage pool? No reasib why not, I suppose.

This machine will then use virtual RAID (which I've no idea about yet) and
aggregate the storage as a volume or how many I need. Next I export this to
the servers via iSCSI for their larger ongoing storage needs.

Pretty much. Once you have the big software RAID stripe, you can use this to back any number of iSCSI volumes.

Note that software RAID only goes up to RAID 6 (i.e. n+2). So you cannot lose more than 2 nodes (FC or iSCSI), otherwise you lose your data.

Now I can use say a small FC GFS target to share various files such as web
pages and other web based shared files yet have the larger more easily
expandable V-RAID for media and other such things.

Once you have a big RAID-ed storage pool, you can partition it out in whatever way you like. You also don't have to put it all into one big RAID stripe, but a few smaller ones.

You can dynamically add disks to a software RAID stripe.

This means that I also need to find an iSCSI target driver that doesn't take
rocket science to figure out and is open source to keep things cheap while
trying this out.

yum install iscsi-target

NFS can give considerably better performance than GFS under some
circumstances. If you don't need POSIX compliant file locking, you may

As I've put all of this together, I've come to learn that I need GFS for the
shared data but the rest, I don't need anything but standard storage. I'll
maintain a cluster of web servers which use GFS to share their pages/images
but I could use this aggregator idea for the larger scale media storage.

Sure, but you aggregator could export space via iSCSI or via NFS, whichever you prefer.

I had somehow gotten a little too caught up in GFS and I was basically not
thinking anything beyond that. This makes things so much simpler than where I
was heading.

It happens to the best of us. :)

I suspect that possibly overkill. It's NIC I/O you'll need more than
anything. Jumbo frames, as big as your hardware can handle, will also help.

Doesn't NIC I/O take up a lot of CPU time?

Not really. Topping out your CPU with NIC I/O load isn't all that trivial. There are also NICs that can offload the entire TCP/IP stack off the CPU onto the NIC, but I don't know that the driver support for Linux is like.

There is no reason why the aggregator couldn't mind it's own exports,
and run as one of the client cluster nodes.

I just mean for fail over. It might be nice to have a little redundancy there.

Sure, you could set up heartbeat and preferably some fencing. I suspect that double activating a software RAID stripe would be quite destructive for your data (I asked about that in a different thread, but nobody stepped up to clarify yet), so fencing is a good idea, "just to make sure". When a node needs to take over, it fences the other node, connects the iSCSI shares, starts up the RAID on them, assumes the floating IP and exports the iSCSI/NFS shares.


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]