[dm-devel] [PATCH 1/1] dm-mpath: Extend path selector interface for supporting Dell EqualLogic path selector

[Jason Shamberger]

Hannes,

Thanks for the feedback.  I looked through the spec you mentioned, and the logical block dependent ALUA functionality is similar to what we are trying to accomplish, however it is not a good fit with our storage array architecture.  Our storage array exposes all LUNs through a single Target Port Group, whereas multiple TPGs are needed for the ALUA functionality.  The design choice of a single TPG was made to simplify the logic on the initiator side.  The client does not need to decide which TPG(s) to connect to, it always connects to the single Target Port Group and the storage array decides where to place the connections through use of iSCSI redirection.

Our goal with this project is to keep the simplicity aspects of our storage array architecture, but add a path selector on the linux initiator to choose the optimal path for each IO.  In this framework, we need additional information to be passed to the path selector to aid it in choosing the optimal path.
 
Jason

-----Original Message-----
From: Hannes Reinecke [mailto:hare at suse.de] 
Sent: Tuesday, June 29, 2010 4:32 AM
To: device-mapper development
Cc: Parind Shah; Jason Shamberger; agk at redhat.com; Narendran Ganapathy; christophe varoqui
Subject: Re: [dm-devel] [PATCH 1/1] dm-mpath: Extend path selector interface for supporting Dell EqualLogic path selector

Narendran Ganapathy wrote:
> This patch extends the dm-path-selector interface to allow path
> selectors to use extra information from the IO request when selecting a
> path.
> 
> Dell EqualLogic and other iSCSI storage arrays use a distributed
> frameless architecture.  In this architecture, the storage group
> consists of a number of distinct storage arrays ("members"), each having
> independent controllers, disk storage and network adapters.  When a LUN
> is created it is spread across multiple members.  The details of the
> distribution are hidden from initiators connected to this storage
> system.  The storage group exposes a single target discovery portal, no
> matter how many members are being used.  When iSCSI sessions are
> created, each session is connected to an eth port on a single member. 
> Data to a LUN can be sent on any iSCSI session, and if the blocks being
> accessed are stored on another member the IO will be forwarded as
> required.  This forwarding is invisible to the initiator.  The storage
> layout is also dynamic, and the blocks stored on disk may be moved from
> member to member as needed to balance the load.
> 
This sounds surprisingly similar to the upcoming 'Logical block dependent'
ALUA state of the upcoming T10 SPC-4.

Which begs the question if
a) is it implemented as such
and
b) if not why not?

And if it _is_ the logical block dependent ALUA state then yes, we
definitely need to update the multipath infrastructure for this.

However, given the good match between the 'REPORT REFERRALS' command
and the device-mapper table definitions I would rather propose to
use the output of 'REPORT REFERRALS' to create / modify the existing
multipath tables.

Currently we have a strict path-only mapping:

3600508b40008ddd70000600000620000 dm-0 HP,HSV300
[size=16G][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=50][active]
 \_ 6:0:6:1 sde 8:64  [active][ready]
\_ round-robin 0 [prio=10][enabled]
 \_ 6:0:5:1 sdd 8:48  [active][ready]

or, in device-mapper output:
0 33554432 multipath 1 queue_if_no_path 0 2 1 round-robin 0 1 1 8:64 100 round-robin 0 1 1 8:48 100

With referrals we just need to adjust the 'start' and 'length' parameter to create _several_
device-mapper tables, eg

0 16777216 multipath 1 queue_if_no_path 0 2 1 round-robin 0 1 1 8:64 100 round-robin 0 1 1 8:48 100
16777217 33554432 multipath 1 queue_if_no_path 0 2 1 round-robin 0 1 1 8:48 100 round-robin 0 1 1 8:64 100

which would route the first half of the resulting multipath device to path '8:64' and the second half
to path '8:48'.

Which I think is far more logical and in match with the current device-mapper architecture.

But no, I don't have a patch for this. I've yet to see an array supporting referrals.
I might be tempted to do something here if I had one ...

And if your firmware does _not_ support referrals I would strongly advise to reimplement your
mechanism in the context of referrals.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		      zSeries & Storage
hare at suse.de			      +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Markus Rex, HRB 16746 (AG Nürnberg)