[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[Linux-cluster] Re: Some ideas on changes to the lvm.sh agent (or new agents).

Great stuff! Much of what you are describing I've thought about in the past, but just haven't had the cycles to work on. You can see in the script itself, the comments at the top mention the desire to operate on the VG level. You can also see a couple vg_* functions that simply return error right now, but were intended to be filled in.

Comments in-line.

On Sep 28, 2007, at 11:14 AM, Simone Gotti wrote:


Trying to use a non cluster vg in redhat cluster I noticed that lvm.sh, to avoid metadata corruption, is forcing the need of only one lv per vg.

I was thinking that other clusters don't have this limitation as they
let you just use a vg only on one node at a time (and also on one
service group at a time).

To test if this was possible with lvm2 I made little changes to lvm.sh
(just variables renames, use of vgchange instead of lvchange for tag
adding) and using the same changes needed to /etc/lvm/lvm.conf
(volume_list = [ "rootvgname", "@my_hostname" ]) looks like this idea
was working.

I can activate the vg and all of its volume only on the node with the vg
tagged with its hostname and the start on the other nodes is refused.

Now, will this idea be accepted? If so these are a list of possible
needed changes and other ideas:

*) Make <parameter name="vg_name" required="1"> also unique="1" or
better primary="1" and remove the parameter "name" as only one service
can use a vg.

Sounds reasonable. Be careful when using those parameters though, they often result in cryptic error messages that are tough to follow. I do checks in lvm.sh where possible to be able to give the user more information on what went wrong.

*) What vg_status should do?
a) Monitor all the LVs
b) Check only the VG and use ANOTHER resource agent for every lv used by
the cluster? So I can create/remove/modify lvs on that vg that aren't
under rgmanager control without any error reported by the status
functions of the lvm.sh agent.
Also other clusters distinguish between vg and lv and they have 2
different agents for them.

This is were things get difficult. It would be ok to modify lvs on that vg as long as it's on the same machine that has ownership. Tags should prevent otherwise, so should be ok.

User would have to be careful (or barriers would have to prevent) users from assigning different LVs in the same VG to different services. Otherwise, if a service fails (application level) and must be moved to a different machine, we would have to find a way to move all services associated with the VG to the next machine. I think there are ways to mandate this (that service A stick with service B), but we would have to have a way to enforce it.

Creating two new agents will also leave the actual lvm.sh without
changes and keep backward compatibility for who is already using it.

Something like this (lets call lvm_vg and lvm_lv respectively the agents
for the vg and the lv):

<service name="foo">
  <lvm_vg vgname="vg01">
     <lvm_lv lvname="lv01/>
     <lvm_lv lvname="lv01/>
     <script .... />

I'd have to be convinced of this... I'd hate to see two 'lvm_lv's that refer to the same VG in different services... or would lvm_lv somehow require a parent lvm_vg, and make lvm_vg have a 'unique=1'? In any case, we are forced to move an entire VG at a time, so I'm not sure it would make sense to break it up.

*) Another problem that is present just now is that lvm should be
changed to avoid any operation on a non activable vg or lv. In these
days you cannot be able to start a vg/lv as its not tagged with the
hostname but you can remove/resize it without any problem. :D

Yes, tags need to be tightened-up. You should never be able to alter metadata of LVM from a machine that does not have permission.

Another improvement I'd like to see is automatic VG ownership. Basically, tighten-up the tagging as stated above and add a unique machine specific tag to new VGs by default. (For clusters, this tag could be the cluster name.) This does a number of things, but primarily:
1) restricts access to the machine (or cluster) that created the VG.
- now if you move a hard drive to a different machine, it won't conflict with the other hard drives there (which might cause confusion over which device is the actual root volume).
2) simplifies HA LVM (lvm.sh + rgmanager)
- there would no longer be a need to edit lvm.conf to add the machine name, etc... One of the questions surrounding this is how do you generate the unique id and when? You would probably need it upon installation...

There are many kinds of fault scenarios. Thinking through each one of them will be important when trying to implement a by-VG model vs a by-LV model. I usually stumble across an issue that gives me pause before hastily adding a new feature, but I would welcome further thought/code.


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]