[Linux-cluster] RHEL 6 cluster filesystem resource and LVM snapshots

Thu Nov 25 19:27:35 UTC 2010

Chris,

2010/11/25 Jankowski, Chris <Chris.Jankowski at hp.com>:
> Brem,
>
> Could you expand a little on your comments, please? I am new to this area and do not quite grasp some of the fine points?
>
> 1.
> Are you saying that I could configure CLVM for exclusive activation for needs of a non-clustered filesystem (ext4/XFS) by using the LVM resource (lvm.sh) now delivered as standard part of Linux Cluster in RHEL6?

I am saying that in order to be able to use non clustered FS's  in a
cluster you would need a mechanism that hardens to a certain level
your data.
2 options are possible:
1) HA-LVM : what was recommended  until RHEL 5 by Redhat and
apparently it's still the case .No need for CLVM in this very case.
see link (even for CLuster3  ie RHEL6)
http://sources.redhat.com/cluster/wiki/LVMFailover. HA-LVM is exactly
what was proposed by HP with SGLX,  tags.

2) Exclusive activation: Requires CLVM . The problem is that RedHat
never provided a resource script to do so. Some have written such
resources and proposed it to the Cluster team but never got
integrated. If you want to use them, there will be no support fro
Redhat(the one from Rafael Miranda worked quite well for me, I even
changed it to manage exclusive activation at VG level, not at LV ).
>
> Specifically we are talking about ext4/XFS filesystems mounted exclusively by one node at a time, of course, and managed by a HA service. This service would contain the LVM resource, filesystem resource, IP resource and script resource.  The physical storage is on FC SAN and shared by all nodes of the cluster and visible to the nodes through multiple paths through Device Mapper Multipath.
>
Classical setup, shared storage (SAN...) active/passive resources,
what most of the clients do.
> 2.
> If so, is a special configuration required for CLVM itself, or is that just the effect of use of the LVM resource?
>
The shipped LVM resource will not allow tyou to do exclusive activation.
> 3.
> In this configuration, would I be able to use filesystem snapshots?
>
Never tried that, do not know, but it seems that some gave you
pointers saying that snapshots are not supported on CLVM.
> 4.
> The --config CLI option you mentioned is an option for what?
>

you can do a lot from the lvm cli (vgchange --config "global
{locking_type=1}" -ay myvg) bypasses the lvm.conf locking_type (=3 if
clustered) and allows you to activate the VG on the wrong node...

> Thanks and regards,
>
> Chris Jankowski
>
> -----Original Message-----
> From: linux-cluster-bounces at redhat.com [mailto:linux-cluster-bounces at redhat.com] On Behalf Of Brem Belguebli
> Sent: Thursday, 25 November 2010 10:07
> To: linux clustering
> Subject: Re: [Linux-cluster] RHEL 6 cluster filesystem resource and LVM snapshots
>
> Hi,
>
> clvm for this specific need, ie using a non clustered filesystem (ext4,
> xfs...) on top of a clustered VG/LV needs the exclusive activation to be
> setup to prevent activation on another node of the running cluster.
>
> The main problem with this is that:
>  *up  until not so far, no RH  cluster shipped lvm resource script
> supported this exclusive activation, despite the fact that there were
> proposals from many people to the RHCS mailing list (look at Rafael
> Miranda proposal around June 2009 in the ml archive)
>  * On RHCS docs, it is said that to use HA-LVM for this specific need of
> non clustered FS. (untill RHEL 5 RHCS 2)
>
> Anyway, either method (HA-LVM or CLVM exclusive activation) can be
> overridden with the --config CLI option, making both options not that
> malevolent action safe.
>
>
>
>
>
> On Wed, 2010-11-24 at 14:27 +0000, Jonathan Barber wrote:
>> On 24 November 2010 09:48, Xavier Montagutelli
>> <xavier.montagutelli at unilim.fr> wrote:
>> > On Wednesday 24 November 2010 09:34:48 Jankowski, Chris wrote:
>> >> Xavier,
>> >>
>> >> Thank you for the explanation.
>> >> This all makes sense.
>> >>
>> >> One more question about one of the documents you pointed me to:
>> >>
>> >> What does this do exactly and why do I need it:
>> >>
>> >> Quote:
>> >>
>> >> 4) Update your initrd on all your cluster machines. Example:
>> >> prompt> new-kernel-pkg --mkinitrd \
>> >>         --initrdfile=/boot/initrd-halvm-`uname -r`.img --install `uname -r`
>> >>
>> >> Unquote
>> >
>> > Caution : the following are only supposition, because I haven't read the
>> > lvm.sh script.
>> >
>> > In step 3 of http://sources.redhat.com/cluster/wiki/LVMFailover , you have to
>> > modify your lvm.conf file, "volume_list" parameter, to filter the VG/LV that can
>> > be activated on a particular host.
>> >
>> > In step 4, they say to create a new initrd : I suppose this step is necesary
>> > to include the modified lvm.conf file inside the initrd, and to NOT activate the
>> > VG located on the shared storage at boot time.
>> >
>> > The lvm.sh script must add or remove the "good" tag (i.e. a tag matching the
>> > hostname of the node running the service) on the fly.
>> >
>> > If someone can confirm or give additional pointers ?
>>
>> That's how I understand it.
>>
>> I've used LVM on RHEL5 *without* clvmd and not had any problems with
>> corruption, etc., but I haven't used snapshots. You have to try
>> really, really hard to break the tagged LVM config from the command
>> line, as the tags prevent activation (which also prevents you from
>> accidentally mounting the FS on different nodes). It's worth knowing
>> about the "--config" argument to the LVM commands, and how to active
>> the LVs from the command line so you can do maintenance to the tagged
>> VG/LVs outside of RHCS:
>> $ lvchange --config "activation { volume_list = [ '@$HOSTNAME' ] }"
>> vg00/test -a y
>>
>> I am not a LVM hacker, so take the following comments with the
>> appropriate caution:
>> If you only ever make changes to the LVM on the active node, I think
>> you'd have to be really unlucky to suffer corruption due to stale LVM
>> metadata. Although if you're carrying out long running tasks like
>> relocating PEs, it might be worth freezing the service (I think this
>> is still probably overly cautious). I think if you start making
>> changes on multiple nodes at the same time, you will suffer badly (but
>> the tags should stop this from happening accidentally).
>>
>> If you are activating LV resources on the basis of their VG, LVM
>> snapshots should survive the resource being relocated between nodes;
>> when the VG is deactivated on the original node, both the original and
>> snapshots LV will be deactivated at the same time, so you won't miss
>> any writes in the snapshot.
>>
>> In my RHEL6 test environment, I just created a two node cluster with
>> cman/clvmd and could create a snapshot on the LVs in a shared VG. This
>> fails under RHEL5. I'm not sure I'd trust it to actually work
>> though...
>>
>> It's probably worth directing some of your questions at the LVM list
>> for a more definitive answer:
>> https://www.redhat.com/mailman/listinfo/linux-lvm
>>
>> PS: I'm sure you will, but you should test it :)
>>
>> Cheers
>>
>> >>
>> >> Regards,
>> >>
>> >> Chris Jankowski
>> > --
>> > Xavier Montagutelli                      Tel : +33 (0)5 55 45 77 20
>> > Service Commun Informatique              Fax : +33 (0)5 55 45 75 95
>> > Universite de Limoges
>> > 123, avenue Albert Thomas
>> > 87060 Limoges cedex
>>
>
>
> --
> Linux-cluster mailing list
> Linux-cluster at redhat.com
> https://www.redhat.com/mailman/listinfo/linux-cluster
>
> --
> Linux-cluster mailing list
> Linux-cluster at redhat.com
> https://www.redhat.com/mailman/listinfo/linux-cluster
>