[dm-devel] [PATCH 1/1] dm-mpath: Extend path selector interface for supporting Dell EqualLogic path selector

[Kiyoshi Ueda]

Hi,

On 06/28/2010 10:53 PM +0900, Narendran Ganapathy wrote:
> This patch extends the dm-path-selector interface to allow path
> selectors to use extra information from the IO request when selecting
> a path.
>
> Dell EqualLogic and other iSCSI storage arrays use a distributed
> frameless architecture.  In this architecture, the storage group
> consists of a number of distinct storage arrays ("members"),
> each having independent controllers, disk storage and network adapters.
> When a LUN is created it is spread across multiple members.
> The details of the distribution are hidden from initiators connected to
> this storage system.  The storage group exposes a single target
> discovery portal, no matter how many members are being used.
> When iSCSI sessions are created, each session is connected to an eth
> port on a single member.  Data to a LUN can be sent on any iSCSI
> session, and if the blocks being accessed are stored on another
> member the IO will be forwarded as required.  This forwarding is
> invisible to the initiator.  The storage layout is also dynamic,
> and the blocks stored on disk may be moved from member to member
> as needed to balance the load.
>
> This architecture simplifies the management and configuration of both
> the storage group and initiators.  In a multipathing configuration,
> it is possible to set up multiple iSCSI sessions to use multiple
> network interfaces on both the host and target to take advantage of
> the increased network bandwidth.  An initiator can use a simple round
> robin algorithm to send IO on all paths and let the storage array
> members forward it as necessary.  However, there is a performance
> advantage to sending data directly to the correct member.
> At the same time, the existing techniques of building a separate
> priority group for paths to each controller does not fit this model,
> because the block ranges may be moved at any time from member to member,
> and it is also acceptable to send IO to any member in the group when
> no direct path exists or there is a path error.
>
> We propose to develop a new path selector to perform this location-based
> routing.  The basic idea is to use knowledge about the current location
> of data to prefer paths directly to the owning member, but fall back to
> use any available path when no direct path is available.
> This submission includes the necessary changes to the dm-path-selector
> interface.  In the current interface, the only information passed
> to the select_path routine is the path_selector struct and the number
> of bytes in the request.  To do location based routing, we need
> the address information of the request.

Interesting.
How can the path-selector know which path owns which location?

I can just see "it's invisible from initiator" from your explanation
above, so I can't understand why the path-selector can choose the best
path which is directly connected to the member owning the location.
It can just guess using the new path-selector parameters below?


> We propose to extend the path selector interface to pass the entire
> request pointer to the 'select_path' / 'start_io' /  'end_io' functions
> so that the path selector can use any information therein to route
> the I/O.
>
> We also propose extending the dm_mpath_io structure used to hold
> information about each I/O to include extra fields for the path
> selector to store I/O specific flags and a timestamp, so the path
> selector can determine the latency of I/Os on different paths and
> that information is passed to the 'select_path' / 'start_io' /  'end_io'
> functions for path selector usage. These additions to the dm_mpath_io
> allow future flexibility in developing algorithms that route IO based
> on other information from the request in the future.

I disagree with passing 'struct request' directly to path-selector.
As I said at http://marc.info/?l=dm-devel&m=123330286327402&w=2,
I think we should keep independency of path-selector from the type of
I/O structure as much as possible, since bio-based targets may want
to use them in the future (e.g. read balancing of dm-mirror).

So I would prefer the way below:
  o define what parameters are needed (now and in the future)
  o pass them to path-selector


Also, please post your path-selector patch if you already have,
so that people can understand your needs much correctly.

Thanks,
Kiyoshi Ueda