[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[Linux-cluster] Getting a SPC-3 compliant PR enabled iSCSI target up and running?



Hi,

I have a testing lab where I'm attempting to get some more experience with LIO and targetcli. Is there an IRC channel where cluster and/or target folks hang out?

I have:

- A 3 node RHEL6.2 cluster with clvmd and GFS2
- A Fedora 16 box (kernel 3.1.6) with LIO/targetcli-2.0rc1.fb3-2 for my shared storage

Is there special configuration needs to be done on the target to enable PR, because it doesn't seem to be working. I'm not able to get fence_scsi working.

I'm seeing registrations, but no reservation.

From one of the RHEL6 cluster nodes, and attempt to do a read-key fails:

# sg_persist -n -i -k -d /dev/sdf
PR in: aborted command

Which generates this message over on my Fedora 16 scsi target:

filp_open(/var/target/pr/
aptpl_086b4b49-8736-45e1-a80c-2ddeb8a5a01e) for APTPL metadata failed

# cat /sys/kernel/config/target/core/iblock_0/store01/pr/res_*
APTPL Bit Status: Disabled
Ready to process PR APTPL metadata..
No SPC-3 Reservation holder
No SPC-3 Reservation holder
0x00000008
No SPC-3 Reservation holder
SPC-3 PR Registrations:
iSCSI Node: iqn.1994-05.com.redhat:226e63cf8cf5,i,0x00023d010000 Key: 0x00000000aa230001 PRgen: 0x00000003
iSCSI Node: iqn.1994-05.com.redhat:93471b6582,i,0x00023d010000 Key: 0x00000000aa230002 PRgen: 0x00000007
iSCSI Node: iqn.1994-05.com.redhat:21d24bc1b670,i,0x00023d010000 Key: 0x00000000aa230003 PRgen: 0x00000005
No SPC-3 Reservation holder
SPC3_PERSISTENT_RESERVATIONS

On the cluster node, this is what my fence_scsi log file looks like:

Jan  5 17:38:45 fence_scsi: [debug] main::do_register_ignore (node_key=aa230003, dev=/dev/sdf)
Jan  5 17:38:45 fence_scsi: [debug] main::do_reset (dev=/dev/sdf, status=0) (cmd=sg_turs /dev/sdf)
Jan  5 17:38:45 fence_scsi: [debug] main::do_register_ignore (err=0) (cmd=sg_persist -n -o -I -S aa230003 -d /dev/sdf)
Jan  5 17:38:45 fence_scsi: [error] main::get_reservation_key (err=11) (cmd=sg_persist -n -i -r -d /dev/sdf)

Running that failing sg_persist gives "PR in: aborted command"

A wireshark packet capture of that command shows:

=== packet generated by sg_persist ===

iSCSI (SCSI Command)
    Opcode: SCSI Command (0x01)
    .0.. .... = I: Queued delivery
    Flags: 0xc1
        1... .... = F: Final PDU in sequence
        .1.. .... = R: Data will be read from target
        ..0. .... = W: No data will be written to target
        .... .001 = Attr: Simple (0x01)
    TotalAHSLength: 0x00
    DataSegmentLength: 0x00000000
    LUN: 0000000000000000
    InitiatorTaskTag: 0x20000000
    ExpectedDataTransferLength: 0x00002000
    CmdSN: 0x000000e2
    ExpStatSN: 0x4c6ecb1b
SCSI CDB Persistent Reserve In
    [LUN: 0x0000]
    [Command Set:Direct Access Device (0x00) (Using default commandset)]
    Opcode: Persistent Reserve In (0x5e)
    .... 0001 = Service Action: Read Reservation (0x01)
    Allocation Length: 8192
    Control: 0x00
        00.. .... = Vendor specific: 0x00
        ..00 0... = Reserved: 0x00
        .... .0.. = NACA: Normal ACA is not set
        .... ..0. = Obsolete: 0x00
        .... ...0 = Obsolete: 0x00

=== response from target ===

iSCSI (SCSI Response)
    Opcode: SCSI Response (0x21)
    Flags: 0x80
        ...0 .... = o: No overflow of read part of bi-directional command
        .... 0... = u: No underflow of read part of bi-directional command
        .... .0.. = O: No residual overflow occurred
        .... ..0. = U: No residual underflow occurred
    Response: Command completed at target (0x00)
    Status: Check Condition (0x02)
    TotalAHSLength: 0x00
    DataSegmentLength: 0x00000062
    InitiatorTaskTag: 0x20000000
    StatSN: 0x4c6ecb1b
    ExpCmdSN: 0x000000e3
    MaxCmdSN: 0x000000f2
    ExpDataSN: 0x00000000
    BidiReadResidualCount: 0x00000000
    ResidualCount: 0x00000000
    Request in: 1
    Time from request: 0.000096000 seconds
    SenseLength: 0x0060
SCSI: SNS Info
    [LUN: 0x0000]
    Valid: 0
    .111 0000 = SNS Error Type: Current Error (0x70)
    Filemark: 0, EOM: 0, ILI: 0
    .... 1011 = Sense Key: Command Aborted (0x0b)
    Sense Info: 0x00000000
    Additional Sense Length: 0
    Command-Specific Information: 00000000
    Additional Sense Code+Qualifier: Invalid Field In Cdb (0x2400)
    Field Replaceable Unit Code: 0x00
    0... .... = SKSV: False
    Sense Key Specific: 000000

(there are more packets in the capture if needed)

My target's saveconfig.json looks like this:

{
  "storage_objects": [
    {
      "attributes": {
        "block_size": 512,
        "emulate_dpo": 0,
        "emulate_fua_read": 0,
        "emulate_fua_write": 1,
        "emulate_rest_reord": 0,
        "emulate_tas": 1,
        "emulate_tpu": 0,
        "emulate_tpws": 0,
        "emulate_ua_intlck_ctrl": 0,
        "emulate_write_cache": 0,
        "enforce_pr_isids": 1,
        "is_nonrot": 0,
        "max_sectors": 1024,
        "max_unmap_block_desc_count": 0,
        "max_unmap_lba_count": 0,
        "optimal_sectors": 1024,
        "queue_depth": 128,
        "task_timeout": 0,
        "unmap_granularity": 0,
        "unmap_granularity_alignment": 0
      },
      "dev": "/dev/vg_station11/iscsi-
lun01",
      "name": "store01",
      "plugin": "block",
      "wwn": "086b4b49-8736-45e1-a80c-2ddeb8a5a01e"
    }
  ],
  "targets": [
    {
      "fabric": "iscsi",
      "tpgs": [
        {
          "attributes": {
            "authentication": 0,
            "cache_dynamic_acls": 0,
            "default_cmdsn_depth": 16,
            "demo_mode_write_protect": 1,
            "generate_node_acls": 0,
            "login_timeout": 15,
            "netif_timeout": 2,
            "prod_mode_write_protect": 0
          },
          "luns": [
            {
              "index": 0,
              "storage_object": "/backstores/block/store01"
            }
          ],
          "node_acls": [
            {
              "attributes": {
                "dataout_timeout": 3,
                "dataout_timeout_retries": 5,
                "default_erl": 0,
                "nopin_response_timeout": 5,
                "nopin_timeout": 5,
                "random_datain_pdu_offsets": 0,
                "random_datain_seq_offsets": 0,
                "random_r2t_offsets": 0
              },
              "chap_mutual_password": "",
              "chap_mutual_userid": "",
              "chap_password": "",
              "chap_userid": "",
              "mapped_luns": [
                {
                  "index": 0,
                  "write_protect": false
                }
              ],
              "node_wwn": "iqn.1994-05.com.redhat:21d24bc1b670",
              "tcq_depth": 16
            },
            {
              "attributes": {
                "dataout_timeout": 3,
                "dataout_timeout_retries": 5,
                "default_erl": 0,
                "nopin_response_timeout": 5,
                "nopin_timeout": 5,
                "random_datain_pdu_offsets": 0,
                "random_datain_seq_offsets": 0,
                "random_r2t_offsets": 0
              },
              "chap_mutual_password": "",
              "chap_mutual_userid": "",
              "chap_password": "",
              "chap_userid": "",
              "mapped_luns": [
                {
                  "index": 0,
                  "write_protect": false
                }
              ],
              "node_wwn": "iqn.1994-05.com.redhat:93471b6582",
              "tcq_depth": 16
            },
            {
              "attributes": {
                "dataout_timeout": 3,
                "dataout_timeout_retries": 5,
                "default_erl": 0,
                "nopin_response_timeout": 5,
                "nopin_timeout": 5,
                "random_datain_pdu_offsets": 0,
                "random_datain_seq_offsets": 0,
                "random_r2t_offsets": 0
              },
              "chap_mutual_password": "",
              "chap_mutual_userid": "",
              "chap_password": "",
              "chap_userid": "",
              "mapped_luns": [
                {
                  "index": 0,
                  "write_protect": false
                }
              ],
              "node_wwn": "iqn.1994-05.com.redhat:226e63cf8cf5",
              "tcq_depth": 16
            }
          ],
          "portals": [
            {
              "ip_address": "10.100.0.12",
              "port": 3260
            }
          ],
          "tag": 1
        }
      ],
      "wwn": "iqn.2003-01.org.linux-iscsi.station11.x8664:sn.32668e1cd52d"
    }
  ]
}

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]