[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Linux-cluster] CLVMD without GFS


If you make a VG non-clustered then you, by definition, forfeit all cluster locking protection on all of the LVs in that group.

In practise you should always make shared volumes clustered and non-shared volumes non-clustered. If you make a shared volume non-clustered then it's up to you to manage the protection of it.

It might be possible to put some protection in for when a volume group is changed from clustered to non-clustered, but really, you're not supposed to do that!

Look at it this way, if you create a shared VG and mark it non-clustered to start with, then you can corrupt it as much as you like by mounting its filesystems on multiple nodes. It's just the same as a SAN volume then.

On 07/21/2009 12:16 PM, brem belguebli wrote:
Hi Chrissie,
Indeed, by default when creating the VG, it is clustered, thus when
creating the LV it is active on all nodes.
To avoid data corruption, I have re created the VG as non clustered
(vgcreate -c n vgXX) then created the LV which got activated only on the
node where it got created.
Then changed the VG to clustered (vgchange -c y VGXX) and activated it
exclusively on this node.
But, I could reproduce the behaviour of bypassing the exclusive flag:
On node B, re change the VG to non clustered though it is activated
exclusively on node A.
and then activate it on node B and it works.
The thing I'm trying to point is that simply by erasing the clustered
flag you can bypass the exclusive activation.
I think a barrier is necessary to prevent this to happen, removing the
clustered flag from a VG should be possible only if the node holding the
VG exclusively is down (does the lock manager  DLM report which node
holds exclusively a VG ?)

2009/7/21, Christine Caulfield <ccaulfie redhat com
<mailto:ccaulfie redhat com>>:


    I've just tried this on my cluster and it works fine.

    What you need to remember is that lvcreate on one node will also
    activate the LV on all nodes in the cluster - it does an implicit
    lvchange -ay when you create it. What I can't explain is why
    vgchange -ae seemed to work fine on node A, it should give the same
    error as on node B because LVs are open shared on both nodes.

    Its not clear to me when you tagged the VG as clustered, so that
    might be contributing to the problem. When I create a new VG on
    shared storage it automatically gets labelled clustered so I have
    never needed to do this explicitly. If you create a non-clustered VG
    you probably ought to deactivate it on all nodes first as it could
    mess up the locking otherwise. This *might* be the cause of your

    The error on clvmd startup can be ignored. It's caused by clvmd
    ussing a background command with --no_locking so that it can check
    which LVs (if any) are already active and re-acquire locks for them

    Sorry this isn't conclusive, The exact order in which things are
    happening is not clear to me.


    On 07/21/2009 10:21 AM, brem belguebli wrote:

        Hi all,
        I think there is something to clarify about using CLVM across a
        in a active/passive mode without GFS.
          From my understanding, CLVM keeps LVM metadata coherent among the
        cluster nodes and provides a cluster wide locking mechanism that can
        prevent any node from trying to activate a volume group if it
        has been
        activated exclusively (vgchange -a e VGXXX)  by another node (which
        needs to be up).
        I have been playing with it to check this behaviour but it
        doesn't seem
        to make what is expected.
        I have 2 nodes (RHEL 5.3 X86_64, cluster installed and
        configured) , A
        and B using a SAN shared storage.
        I  have a LUN from this SAN seen by both nodes, pvcreate'd
        /dev/mpath/mpath0 , vgcreate'd vg10 and lvcreate'd lvol1 (on one
        created an ext3 FS on /dev/vg10/lvol1
        CLVM is running in debug mode (clvmd -d2 ) (but it complains about
        locking disabled though locking set to 3 on both nodes)
        On node A:
                   vgchange -c y vg10 returns OK (vgs -->  vg10     1
        1   0
                   vgchange -a e --> OK
                   lvs returns lvol1   vg10   -wi-a-
        On node B (while things are active on A, A is UP and member of the
        cluster ):
                   vgchange -a e --> Error locking on node B: Volume is
        busy on
        another node
                                            1 logical volume(s) in
        volume group
        "vg10" now active
        It activates vg10 even if it sees it busy on another node .
        on B, lvs returns lvol1   vg10   -wi-a-
        as well as on A.
        I think the main problem comes from the fact that, as it is said
        starting CLVM in debug mode,  WARNING: Locking disabled. Be careful!
        This could corrupt your metadata.
        IMHO, the algorithm should be as follows:
        VG is tagged as clustered (vgchange -c y VGXXX)
        if a node (node P) tries to activate the VG exclusively
        (vgchange -a VGXXX)
        ask the lock manager to check if VG is not already locked by another
        node (node X)
        if so, check if node X is up
        if node X is down, return OK to node P
        return NOK to node P (explicitely that VG is held exclusively by
        node X)
        PS: this shouldn't be a problem with GFS or other clustered FS
        etc...) as no node should try to activate exclusively any VG.


        Linux-cluster mailing list
        Linux-cluster redhat com <mailto:Linux-cluster redhat com>

    Linux-cluster mailing list
    Linux-cluster redhat com <mailto:Linux-cluster redhat com>


Linux-cluster mailing list
Linux-cluster redhat com

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]