[linux-lvm] LVM + raid + san

[Stuart D. Gathman]

On Sat, 6 Nov 2010, Phillip Susi wrote:

> My understanding of a SAN is where you get a few drive enclosures and a few
> servers and plug them all into a sas expander so all of the servers can see
> all of the disks.  You seem to be talking about having all of the disks on one
> server that then serves them over ethernet with iscsi.  I wouldn't want to do
> that because it adds a good deal of overhead to the disk access and introduces
> a single point of failure.

Your idea is not typical for SAN, where the point is to centralize dealing with
physical disks. The SAN server can be internally highly redundant, addressing
the single point of failure issue.  

However, thinking how to do what you want, how about getting enclosures with
built in RAID?  For instance, There are low cost NAS and USB enclosures with
built in RAID-1.  Perhaps there are some more expensive ones with RAID 10.
(I still wouldn't use RAID5).  Then you can just use them with LVM without
as much worry about disk failure.  If an enclosure fails, however, that
PV will be offline (disrupting your service) until you replace it (moving
disks to new enclosure - so you don't lose data, other than the chaos from
any LVs that are partially offline).

> I'd rather just use LVM to manage all of the disks as part of a single volume
> group so you can immediately transfer a lv from one server to another, but I
> can't work out how to still manage to get raid without having lvm do it with
> the dm-raid5 support.

You'd have some serious locking issues.  With RAID5, each server would have to
lock each chunk before writing to it (which involves a read/modify/write
cycle).  This would create serious overhead.  And you were complaining about
SAN server overhead! :-)  RAID5 *must* be centralized.  Your scheme might work
with RAID10, but then you'd still have to ensure that writes to mirror legs
don't get out of order, with updates from multiple servers flying over the
wire.

I still think you want a traditional SAN with an enterprise SAN server with
lots of built in redundancy (or build your own .  You'll help feed the family
of a hard working salesman :-)

-- 
	      Stuart D. Gathman <stuart at bmsi.com>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.