[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Linux-cluster] CLVM clarification



Sorry, but I have not clear the clvm role.

CLVM shares VG metadata over a cluster and makes possible a wide cluster administration (RedHat documentation says).

In this way a CLVM Cluster must have a CMAN Cluster up and running.

So, if I have already a shared storage, the only thing I can do is to make a GFS filesystem and export this one to clients machine. In this way the shared storage could be accessed by multiples machines.

In this scenario clvm is not useful because the shared lock on filesystem is guaranted by GFS.

Let's pose I have different machines that I want to join in a cluster. Each machine has a storage that I want share with the other machines to create a large storage.

With CLVM, stands to RedHat guide, I can create a cluster that "presenting each cluster computer with the same view of the logical volumes".[1]

So I have:

node 1:
  VG1  (local)
  VG2  (node2)
  VG3  (node3)

node 2:
  VG1  (node1)
  VG2  (local)
  VG3  (node3)

node 3:
  VG1  (node1)
  VG2  (node2)
  VG3  (local)

This should be what the RedHat CLVM guide stands for "the same view of the logical volumes"

From this point, Node1 is the shared storage. In this example it is visible from alle the cluster's nodes.

So if I stroke an "lvcreate", I have to see the newly create LV on the other nodes of the cluster.
It is true?

If this is true, gndb is not necessary and the layout becomes really simple.

Thanks for your time!


2008/10/2 Xavier Montagutelli <xavier montagutelli unilim fr>
On Thursday 02 October 2008 12:28, Angelo Compagnucci wrote:
> Ok, this could be clear, but in the Cluster_Logical_Volume_Manager.pdf I've
> read (bottom of page 3):
> "The clmvd daemon is the key clustering extension to LVM. The clvmd daemon
> runs in each cluster computer and distributes LVM metadata updates in a
> cluster, presenting each cluster computer with the same view of the logical
> volumes"
>
> This is a picture of wath I have in mind:

This picture doesn't show the difference between a GNBD server (which doesn't
know anything about the use of the exported block device : it doesn't know
the VG for example) and the GNBD clients (which actually use the block device
as PV). May I add some layers ? Not exactly what I have in mind but I am not
a ascii art expert :

 ---------------------------
|      GFS filesystem       |
 ---------------------------
|            LV             |
 ---------------------------
|            VG             |
 ---------------------------
|  PV1    |  PV2   |   PV3  |
.---------.--------.--------.
|             CLVM          |
.---------.--------.--------.
|  cluster basis (dlm,...)  |
.---------.--------.--------.
| Node4   | Node5  | Node6  |
.---------.--------.--------.
(Node4,5,6 have access to the three GNBD devices)
       \    | |    /
        \___|_|___/
        /   | |   \
       /    | |    \
      /     | |     \
.---------.--------.---------.
| GNBD1   | GNBD2  | GNBD3   |
.---------.--------.---------.
| hda1    |  hda1  |   hda1  |
| Node1   | Node2  |   Node3 |
.---------.--------.---------.

>
> In this case the clvm features are not useful because there is only one
> machine (that could not be a node of a cluster) that have the lvm over GNBD
> exported devices. So the nodes doesn't know nothing about the other nodes.

If your GNBD* devices are accessed by only one other node. But if the GNBD are
served to multiple nodes (nodes4,5,6), then CLVM is useful.

>
> Let's pose this situation:
>
> -----------------------------------------------
> |            GFS                           |
> -----------------------------------------------
> |                LV                          |
> -----------------------------------------------
> |         VG1         |        VG2      |
> -----------------------------------------------
> |         PV1         |        PV2       |
> |      Node1         |      Node2      |
> -----------------------------------------------
> |        CLVM coordinates           |
> -----------------------------------------------
>
> In this situatuation makes sense to have a clustered lvm because if I have
> to make some maintenance over VGs, CLVM can lock and unlock the interested
> device.
>
> Is this the correct behaviour??

Perhaps I miss your point, but it doesn't make sense if the block devices are
local to each node. How could Node2 have access to the block device on Node1
(showed as PV1) ?

CLVM is useful only when you have a shared storage.

> In the contrary, which is the CLVM role in a cluster?

>From what I know, CLVM protects the metadata parts of LVM on the shared
storage. And when you make one operation the shared storage on one node (for
example, create a new LV), all the nodes are aware of the change.


>
>
> 2008/10/2 Xavier Montagutelli <xavier montagutelli unilim fr>
>
> > On Wednesday 01 October 2008 17:39, Angelo Compagnucci wrote:
> > > Hi to all,This is my first post on this list. Thanks in advance for
> > > every answer.
> > >
> > > I've already read every guide in this matter, this is the list:
> > >
> > > Cluster_Administration.pdf
> > > Cluster_Logical_Volume_Manager.pdf
> > > Global_Network_Block_Device.pdf
> > > Cluster_Suite_Overview.pdf
> > > Global_File_System.pdf
> > > CLVM.pdf
> > > RedHatClusterAdminOverview.pdf
> > >
> > > The truth is that I've not clear a point about CLVM.
> > >
> > > Let's me make an example:
> > >
> > > In this example CLVM and the Cluster suite are fully running without
> > > problems. Let's pose the same configuration of cluster.conf and
> > > lvm.conf and the nodes of the cluster are joined and operatives.
> >
> > Does your example include a shared storage (GNBD, iSCSI, SAN, ...) ?
> >
> > > NODE1:
> > >
> > > pvcreate /dev/hda3
> > >
> > > NODE2:
> > >
> > > pvcreate /dev/hda2
> > >
> > > Let's pose that CLVM spans LVM metadata across the cluster, if I stroke
> >
> > the
> >
> > > command:
> > >
> > > pvscan
> > >
> > > I should see /dev/sda2 and /dev/sda3
> > >
> > > and then I can create a vg with
> > >
> > > vgcreate /dev/sda2 /dev/sda3 ...
> > >
> > > The question is: How LVM metadata sharing works? I have to use GNBD on
> >
> > the
> >
> > > row partion to share a device between nodes? I can create a GFS over a
> > > spanned volume group? Are shareable only logical volumes?
> >
> > I have the feeling that something is not clear here. I am not an expert,
> > but :
> >
> > GNBD is just a mean to export a block device on the IP network. A GNBD
> > device
> > is accessible to multiple nodes at the same time, and thus you can
> > include that block device in a CLVM Volume Group. Instead of GNBD, you
> > can also use any other shared storage (iSCSI, FC, ...). Be careful, from
> > what I have understood, some SAN storage are not sharable between many
> > hosts (NBD, AoE for example) !
> >
> > After that, you have the choice :
> >
> >  - to make one LV with a shared filesystem (GFS). You can then mount the
> > same
> > filesystem on many nodes at the same time.
> >
> >  - to make many LV with an ext3 / xfs / ... filesystem. But you then have
> > to
> > make sure that one LV is mounted on only one node at a given time.
> >
> > But the type of filesystem is independant, this is a higher component.
> >
> > In this picture, CLVM is only a low-level component, avoiding the
> > concurrent
> > access of many nodes on the LVM metadata written on the shared storage.
> >
> > The data are not "spanned" across the local storage of many nodes (well,
> > I suppose you *could* do that, but you would need other tools / layers ?)
> >
> > Other point : if I remember correctly, the Red Hat doc says it's not
> > recommended to use GFS on a node that exports a GNBD device. So if you
> > use GNBD as a shared storage, I suppose it's better to specialize one or
> > more nodes as GNBD "servers".
> >
> >
> > HTH
> >
> > > Thanks for your answers!!
> >
> > --
> > Xavier Montagutelli                      Tel : +33 (0)5 55 45 77 20
> > Service Commun Informatique              Fax : +33 (0)5 55 45 75 95
> > Universite de Limoges
> > 123, avenue Albert Thomas
> > 87060 Limoges cedex
> >
> > --
> > Linux-cluster mailing list
> > Linux-cluster redhat com
> > https://www.redhat.com/mailman/listinfo/linux-cluster

--
Xavier Montagutelli                      Tel : +33 (0)5 55 45 77 20
Service Commun Informatique              Fax : +33 (0)5 55 45 75 95
Universite de Limoges
123, avenue Albert Thomas
87060 Limoges cedex

--
Linux-cluster mailing list
Linux-cluster redhat com
https://www.redhat.com/mailman/listinfo/linux-cluster


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]