[dm-devel] dm-multipath + iscsi-initiator-core success, example, and questions
Nicholas A. Bellinger
nick at pyxtechnologies.com
Thu Mar 31 23:52:11 UTC 2005
Greetings all,
While doing testing and gathering data for dm+multipath
+iscsi-initiator-core documentation, I had a few questions as to when
paths are considered offline when a path is blocked, and online when a
path is unblocked.
A little background on iSCSI Channels first: An iSCSI Channel maps to a
struct Scsi_Host within the SCSI Midlayer. It is an abstraction for
iSCSI Initiator management purposes that can keep track of IO and
settings once an iSCSI Session that is attached to said iSCSI Channel
has been logged out, or failed due to various exceptions.
Now, consider the examples using two iSCSI Channels defined
in /etc/sysconfig/initiator:
CHANNEL="20 1 eth0 192.168.0.60 3260 0 AuthMethod=CHAP,None"
CHANNEL="21 1 eth0 192.168.1.60 3260 0 AuthMethod=CHAP,None"
iSCSI Channels 20 and 21 are pointing to the same iSCSI Target Node, but
to different iSCSI Target Portal Groups with the _SAME_ LUN view behind
both Target Portals.
Initiator Node Target Node
------------ ---------------------
| Channel 20 | <----> | TPG 0 : 192.168.0.60| \
| | | | [LUN 0]
| Channel 21 | <----> | TPG 1 : 192.168.1.60| /
------------ ---------------------
iSCSI Sessions with a single iSCSI Connection are started on Channels 20
and 21 with the following:
/etc/rc.d/init.d/initiator start 20
iCHANNEL[20] - Allocated Linux SCSI Host with ID: 4
Received OK response from PyX Authentication Daemon, continuing.
Received OK response from PyX Authentication Daemon, continuing.
iSCSI mutual authetication completed successfully.
iCHANNEL[20] - iSCSI login successful on CID: 0 to 192.168.0.60:3260,0
iCHANNEL[20] - Incremented iSCSI connection count to 1 to node:
iqn.2002-07.com.pyxtechnologies.target:sn.114364923233
iCHANNEL[20] - Established iSCSI session to node:
iqn.2002-07.com.pyxtechnologies.target:sn.114364923233
iSCSI Core Stack[1] - Incremented number of active iSCSI sessions to 1.
Vendor: SEAGATE Model: ST373454LC Rev: 0002
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sdb: 143374744 512-byte hdwr sectors (73408 MB)
SCSI device sdb: drive cache: write back
SCSI device sdb: 143374744 512-byte hdwr sectors (73408 MB)
SCSI device sdb: drive cache: write back
sdb: unknown partition table
(... other output removed for example purposes)
/etc/rc.d/init.d/initiator start 21
iCHANNEL[21] - Allocated Linux SCSI Host with ID: 5
Received OK response from PyX Authentication Daemon, continuing.
Received OK response from PyX Authentication Daemon, continuing.
iSCSI mutual authetication completed successfully.
iCHANNEL[21] - iSCSI login successful on CID: 0 to 192.168.1.60:3260,1
iCHANNEL[21] - Incremented iSCSI connection count to 1 to node:
iqn.2002-07.com.pyxtechnologies.target:sn.114364923233
iCHANNEL[21] - Established iSCSI session to node:
iqn.2002-07.com.pyxtechnologies.target:sn.114364923233
iSCSI Core Stack[1] - Incremented number of active iSCSI sessions to 2.
iSCSI Core Stack[1] - Incremented number of active iSCSI sessions to 2.
Vendor: SEAGATE Model: ST373454LC Rev: 0002
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sdh: 143374744 512-byte hdwr sectors (73408 MB)
SCSI device sdh: drive cache: write back
SCSI device sdh: 143374744 512-byte hdwr sectors (73408 MB)
SCSI device sdh: drive cache: write back
sdh: unknown partition table
(... other output removed for example purposes)
Consider the following simple table for dm-multipath, piped directly
into dmsetup:
echo "0 143374744 multipath 0 0 2 1 round-robin 0 2 1 8:16 1000 8:112
1000 round-robin 0 2 1 8:16 1000 8:112 1000" | dmsetup create dm0
Major/Minor 8:16 corresponds to iSCSI Channel 20, LUN 0 (/dev/sdb)
Major/Minor 8:112 corresponds to iSCSI Channel 21, LUN 0 (/dev/sdh)
Now consider the following example iSCSI management scenario while doing
READs to device-mapper:
lmdd if=/dev/mapper/dm0 of=/dev/null bs=4096
Now lets pause iSCSI Channel 20: (Note that pausing an iSCSI Channel
puts the attached struct Scsi_Host's struct scsi_devices into a blocked
state with scsi_internal_device_block()).
/etc/rc.d/init.d/initiator pause 20
iCHANNEL[20] - Stopped session on iSCSI Channel
iCHANNEL[20] - Stopped all action on iSCSI Channel
iCHANNEL[20] - Decremented iSCSI connection count to 0 to node:
iqn.2002-07.com.pyxtechnologies.target:sn.114364923233
SCSI error : <4 0 0 0> return code = 0x20000
end_request: I/O error, dev sdb, sector 9293248
device-mapper: dm-multipath: Failing path 8:16.
end_request: I/O error, dev sdb, sector 9293256
SCSI error : <4 0 0 0> return code = 0x20000
end_request: I/O error, dev sdb, sector 9293504
end_request: I/O error, dev sdb, sector 9293512
iCHANNEL[20] - released iSCSI session to node:
iqn.2002-07.com.pyxtechnologies.target:sn.114364923233
iSCSI Core Stack[1] - Decremented number of active iSCSI sessions to 1
Data on iSCSI Channel 21 continues to flow, but even once iSCSI Channel
20 is restarted with:
/etc/rc.d/init.d/initiator start 20 (Note that starting a paused iSCSI
Channel puts the attached struct Scsi_Host's struct scsi_devices into a
unblocked state with scsi_internal_device_unblock()).
iCHANNEL[20] - Allocated Linux SCSI Host with ID: 4
Received OK response from PyX Authentication Daemon, continuing.
Received OK response from PyX Authentication Daemon, continuing.
iSCSI mutual authetication completed successfully.
iCHANNEL[20] - iSCSI login successful on CID: 0 to 192.168.0.60:3260,0
iCHANNEL[20] - Incremented iSCSI connection count to 1 to node:
iqn.2002-07.com.pyxtechnologies.target:sn.114364923233
iCHANNEL[20] - Established iSCSI session to node:
iqn.2002-07.com.pyxtechnologies.target:sn.114364923233
iSCSI Core Stack[1] - Incremented number of active iSCSI sessions to 2.
IO never goes back across iSCSI Channel 20. I also noticed that this
does not occur every time iSCSI Channel 20 is paused. Sometimes IO will
stop over iSCSI Channel 21 as well, but iSCSI Channel 20 can be started
and IO resumes correctly to both iSCSI Channels.
So my main question is this, once the iSCSI Channel is restarted and IO
is allowed to flow again, how does dm-multipath know that the device is
unlocked and online?
Thanks!
--
Nicholas A. Bellinger <nick at pyxtechnologies.com>
More information about the dm-devel
mailing list