[linux-lvm] LVM in shared parallel SCSI environment

[Jos Visser]

Hi Jesse,

Let's see if I get this right: You have an LVM configuration on a shared
SCSI disk set. I understand from your description that you have some
VG's active on more than one node at a time. Is this right? If so, I
wonder if it's supported (others are better equiped to determine this
than I am). However, most volume managers on other Unix platforms do not
allow this. 

As far as I know the vgchange inactive/active bounce is the only thing
that will refresh the metadata. 

However, if you have the VG truly active on more than one node (if
that's possible), you have a recipe for disaster! What if you by mistake
change the VG (e.g. adding an LV) from one node, and then perform a
similar action from another node? I would not be surprised if there
would be corruptions, oopses and panics all over the place.

++Jos

And thus it came to pass that Jesse Sipprell wrote:
(on Sat, Nov 11, 2000 at 08:46:02AM -0500 to be exact)

> Hi all,
> 
> I'm currently in the process of planning a large(ish)-scale Linux deployment.
> As part of the planning process, I am considering using LVM in a shared SCSI
> environment; potentially with GFS as the file system in the future, but
> starting with ext2.
> 
> To that end I have a test "cluster" set up.  It consists of:
> 
> One Winchester Systems OpenRAID chassis with 100GB total storage, running in
> RAID5.  The OpenRAID is a SCSI-to-SCSI solution, with four host SCSI channels.
> The hardware can be configured so that the channels can share physical storage
> space.
> 
> Four test Linux boxes, each with a small (9GB) boot/root/swap/emergency SCSI
> disk.  Obviously, the four boxes are connected to the four host channels on
> the OpenRAID.
> 
> The OpenRAID is configured such that all available disk space is visable
> identically to the servers.  Obviously, because the initial test uses ext2
> instead of a filesystem that supports shared media (i.e. GFS), we won't be
> actually sharing filesystems between boxes.  However, LVM seemed like a pretty
> good potential in this situation because it allows for relatively dynamic
> allocation of storage from a large shared "pool."  The catch, as with all
> shared SCSI solutions, is that each participating host must maintain a
> consistant view of LVM metadata.
> 
> So far, it's been fairly successful.  Once the vg and lvs are created, and each
> host has meta in core, all is well.  The only problem I am seeing is when an
> lv (for example) is extended or reduced, the other systems are unaware of the
> change in LVM metadata.  This makes sense of course, because each host is
> operating with it's own "notion" of the LVM in core.  The solution is rather
> awkward; one has to unmount all LVM filesystems on each host, vgchange the vg
> to inactive, vgchange it back to active and then remount the filesystems.
> 
> My question is this:  Is there any way to "refresh" the in-core metadata from
> disk?  If not, does anyone think this might be a good idea for the future?
> 
> Regards,
> 
> -- 
> Jesse Sipprell
> Technical Operations Director
> Evolution Communications, Inc.
> 800.496.4736
> 
> * Finger jss at evcom.net for my PGP Public Key *

-- 
Success and happiness can not be pursued; it must ensue as the 
unintended side-effect of one's personal dedication to a course greater 
than oneself.