[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Linux-cluster] GNBD and CLVM and multipath oh my


I tested a multipath gnbd setup some weeks ago and posted it to the list on April 19th.
(relevant part of the thread attached)
It works, but it seems to be very experimental :-)



- SAN (sda1+sdb1)
- 2 Nodes directly attached which  export  sda1+sdb1  via gnbd each
(sda1 and sdb1 form a striped lvm)
- Nodes in the LAN which gnbd-import  sda1+sdb1 from each node
-> noda_sda1 as gnbd0
-> noda_sdb1 as gnbd1
-> nodb_sda1 as gnbd2
-> nodb_sdb1 as gnbd3
- now I created a failover multipath configuration
echo "0 85385412 multipath 0 0 2 1 round-robin 0 1 1 251:0 1000
round-robin 0 1 1 251:2 1000" | dmsetup create dma

echo "0 85385412 multipath 0 0 2 1 round-robin 0 1 1 251:3 1000
round-robin 0 1 1 251:1 1000" | dmsetup create dmb

In this configuration traffic to sda1 goes primaly to noda and traffic to sdb1 primaly to nodeb. I adapt lvm.conf not to include /dev/gnbd in the search for volumgroups, instead /dev/mapper/dm (I get rid of the duplicate volumgroup with this workaround). After I start clvmd, I can see the Volume on the client.

With this solution, I have a speedup of about 50% compared to example one
(I think because the striping is done by the client, whereas in example
one the client performs round-robin load-balancing
about differnt pathes and the gnbd server stripes on both disks...)

dmsetup  message dma 0  disable_group 1
dmsetup  message dmb 0  disable_group 2
dmsetup  message dma 0  enable_group 1
dmsetup  message dmb 0  enable_group 2
I can switch between the two pathes.

It will be a bit of work is to get the startup scripts work correctly,
because the dmsetup multipath command depends on the major and minor
device ID's of the gnbd-devices of the client, which seem not to bee

David Golden wrote:

On 2005-06-02 17:29:04 -0500, Benjamin Marzinski wrote:
> First, GNBD does not currently work with multipath. The multipath tools
> are too SCSI-centric. Sorry.

Oh well.

> either (a) or (b) should work. I would bet that (b) gets you better performance
> on the worker nodes. gnbd has one server per exported device per client.

Well, I'll try (b) so. The head nodes are near-dedicated to file server duty and better performance is good :-)

Thanks for your help!

Best Regards,

David Golden


Linux-cluster mailing list
Linux-cluster redhat com

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]