[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Linux-cluster] help on configuring a shared gfs volume in a load balanced http cluster

Hi Gordan,

Thanks for your reply... I have performed some small steps forward...

> > Actually, i want just to have hdb1 from shd1 and hdc1 from shd2 joided in
> > one volume. No mirror for this volume at that momment. Is possible? If
> > yes, how? Using ATAoE?
> Set up ATAoE on shd and use it to export a volume. Connect to this ATAoE
> share from the front end nodes. You can then use something like Cluster
> LVM (CLVM) to unify them into one volume.

Now, I setup ATAoE inside shd1 and shd2 and now, i am able to see exported 
disks on my redhat cluster manager machine (rhclm), so should be the same 
view on all our rs1, rs2 and rs3 webservers.

See below:

[root rhclm ~]# cat /proc/partitions
major minor  #blocks  name

   3     0   19551168 hda
   3     1      98248 hda1
   3     2          1 hda2
   3     5    1000408 hda5
   3     6    8673808 hda6
 152   560   78150744 etherd/e2.3
 152   561   39075088 etherd/e2.3p1
 152   562   39075624 etherd/e2.3p2
 152   288   39082680 etherd/e1.2
 152   289   19541056 etherd/e1.2p1
 152   290   19541592 etherd/e1.2p2

[root rhclm ~]# aoe-stat
      e1.2        40.020GB   eth0 up
      e2.3        80.026GB   eth0 up
[root rhclm ~]#

[root rhclm ~]# ls -l /dev/etherd/*
brw-r----- 1 root disk 152, 288 Jul 25 17:47 /dev/etherd/e1.2
brw-r----- 1 root disk 152, 289 Jul 25 17:47 /dev/etherd/e1.2p1
brw-r----- 1 root disk 152, 290 Jul 25 17:47 /dev/etherd/e1.2p2
brw-r----- 1 root disk 152, 560 Jul 25 17:47 /dev/etherd/e2.3
brw-r----- 1 root disk 152, 561 Jul 25 17:47 /dev/etherd/e2.3p1
brw-r----- 1 root disk 152, 562 Jul 25 17:47 /dev/etherd/e2.3p2
[root rhclm ~]#

So, is exacly what i exported from shd1 and shd2 servers (hdb -> 2*20GB slices 
and hdc -> 2*40GB slices)

>From here i am lost. I understand from you, that now, on one of our 
realservers, let say rs1, shoud i use CLVM to unify them. I want to obtain 2 


I am not sure how to do it using clvm... All is crossing my mind is mdadm...

mdadm -C /dev/md0 -l 0 -n 2 \
        /dev/etherd/e1.2p1 /dev/etherd/e2.3p1

and after that, to use:

pvcreate /dev/md0
vgcreate extendible_lvm /dev/md0

lvcreate --extents 60GB --name extendible_lvm www_docroot

and finally mkfs.gfs /dev/extendible_lvm/www_docroot

Can you help me regading clvm?


> Then create GFS on this volume.
> Note that if you lose either of the two shd machines you will likely lose
> all the data.
> > After that, i would like to know, how to install GFS on this volume and
> > use it as documennt root on our real web servers (rs1, rs2, rs3). Is
> > possible? If yes, how?
> Yes, when you have the logical volume consisting of shd1 and shd2, create
> the GFS on it as per the docs (mkfs.gfs), mount it to where you want it,
> and point Apache at that path. Nothing magical about it, it's just like
> any other once you have it mounted.
> > I don't understand from your explanation, how to group machines: shd1 and
> > shd2 should be in one cluster and rs1, rs2 and rs3 in other cluster
> I don't see why you need shd1 and shd2 machines in a cluster. They are
> just SANs. Unless they are mirroring each other or beign each other's
> backup there is no immediately obvious reason from your example why they
> should be clustered together.
> > or: shd1 and shd2 shoud be regular servers which is just exporting their
> > HDD
> Yes. And you don't export the HDD per se using ATAoE or iSCSI - you export
> a "volume" (which is just a file on shd's file system that is effectively
> a disk image).
> > using ATAoE and rs1, rs2 and rs2 to be grouped in one cluster which
> > are importing a GFS volume from somwhere?
> rs machines would import the ATAoE volumes, establish a logical volume on
> top of them, and then start the GFS file system on top of that.
> > If yes, from where? How can i configure a GFS volume on
> > ATAoE disks and from where will be accesible?
> It will be accessible from any machine in the cluster the GFS volume is
> built for (in this case rs set), once they connect the ATAoE (or iSCSI if
> that's what you use for it, there isn't THAT much difference between
> them) shares from shds.
> > I need another one machine
> > which will act as agregator for ATAoE disks or our real web servers (rs1,
> > rs2, rs3) will responsible to import directly these disks?
> You don't need an agregator, you can unify the volumes using CLVM into one
> big logical volume, and have GFS live on top of that.
> >> and ATAoE or iSCSI to export the
> >> volumes for the rs machines to mount.
> >
> > In our lab we are using regular hard disks, so iSCSI is excluded.
> iSCSI is a network protocol, nothing to do with SCSI disks per se.
> It's SCSI-over-ethernet. You can export any file on a machine as a volume
> using iSCSI. Whether the underlying disk is SCSI, ATA or something exotic
> is entirely irrelevant.
> ATAoE and iSCSI are both applicable to your case. ATAoE has somewhat lower
> overheads (read: a little faster) but is ethernet layer based. iSCSI is
> TCP based so is routable. iSCSI is also a little more mature.
> > I read an article here (http://www.linuxjournal.com/article/8149) about
> > ATAoE and i have some questions:
> >
> > - on our centos 5.2 boxes, we already have aoe kernel module but we don't
> > have aoe-stat command. Is any packet shoud i install via yum to have this
> > command (or other command to hadle aoe disks) or is required do download
> > aoetools-26.tar.gz and compile from source
> > (http://sourceforge.net/projects/aoetools/)
> >
> > - in above article they are talking about RAID10, LVM and JFS. They are
> > not teaching me about GFS and clustering. They choose JFS and not GFS
> > saying that "JFS is a filesystem that can grow dynamically to large
> > sizes, so he is going to put a JFS filesystem on a logical volume". I
> > want that but using GFS, is possible or not?
> There are several concepts and technologies you need to go read up on
> before getting further with this:
> LVM/CLVM for volume management
> If you add additional volumes (e.g. exported via iSCSI or ATAoE) to your
> SAN boxes, you can add them into your CLVM volume you have GFS on top of,
> and the virtual "disk" (logical volume) will show as being bigger. You can
> then grow the GFS file system on this volume and have it extend onto the
> additional space.
> > They are saying that:
> >
> > "using a cluster filesystem such as GFS, it is possible for multiple
> > hosts on the Ethernet network to access the same block storage using ATA
> > over Ethernet. There's no need for anything like an NFS server"
> NFS and GFS are sort of equivalent, layer wise.
> > "But there's a snag. Any time you're using a lot of disks, you're
> > increasing the chances that one of the disks will fail. Usually you use
> > RAID to take care of this issue by introducing some redundancy.
> > Unfortunately, Linux software RAID is not cluster-aware. That means each
> > host on the network cannot do RAID 10 using mdadm and have things simply
> > work out."
> What they are saying is that you can't export two ATAoE/iSCSI shares, have
> mdadm RAID on top, and then have GFS on top, because the mdadm layer isn't
> cluster aware. But you aren't using RAID on that level.
> RAID would be on the shd machines (hardware or mdadm RAID on the disks you
> use for storage, before any exporting via ATAoE or iSCSI happens.
> If you want the servers mirrored (i.e. RAID1), that's what you would use
> DRBD as I mentioned earlier. But then you wouldn't mount a share from each
> machine, you'd mount just one of the two, and have shds clustered for
> fail-over.
> > So, finally, what should i do? Can you or anybody suggest me some howtos
> > and what is the correct order to group machines and implement clustering?
> See above. Have a Google around for the things I mentioned, and ask more
> specific questions. :)
> Gordan
> --
> Linux-cluster mailing list
> Linux-cluster redhat com
> https://www.redhat.com/mailman/listinfo/linux-cluster

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]