[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [dm-devel] Multipath setup and rdac questions

Christian Røsnes wrote:

I have got a couple of MD3000i with dual controllers, and two M600e
blades connected two both of them.

I've setup multipath, and would appreciate if someone could tell me
whether the setup described below looks correct, or if you'd do it
any other way. The reason I'm asking, is that I cannot see any reference to 'rdac hardware handler' when I do "multipath -ll".

However, when I change the module controller ownership one of the
LUNs (while copying files to that particular LUN), the failover
seems to work - the gilr copying continues after the controller

I also see these messages in /var/log/messages when I change the
module controller ownership:

494 [RAIDarray.mpp]MD3000i_A:0:1:0 Cmnd-failed try alt ctrl 1.
vcmnd SN 351494 pdev H2:C0:T0:L0 0x05/0x94/0x01 0x08000002

Don't know where this is coming form. Do you have IBM's MPP
multipathing driver installed? If you are using device-mapper
multipath and multipath tools (the rest of your mail suggests this is
your intention) you should not load the MPP modules.

Q1: What could be the reason that I'm not using the rdac hwhandler
? (see output from "multipath -ll" below)

Your vendor/product details don't seem to match what the array is

Q2: Is using rdac for this kind of setup preferred ?

Yes. It's required for proper failover if you're using device-mapper

Q3: I've created a GPFS filesystem using the /dev/dm-0, /dev/dm-1,
etc device. Is it OK to use the dm-x devices ? I see that there are
other alternatives such as /dev/mapper/mpath<x>.

I always recommend using the /dev/mapper/* entries. These are
guaranteed to exist when the device is created (unlike the
udev-managed symlinks sometimes found in /dev/mpath) and are more
meaningful than the "dm-N" names which are really an internal
device-mapper name for the device.

However, /dev/mapper/mpath<x> is not visible in /proc/partitions (see below), but the dm-<x> devices are - and the GPFS filesystem
complains it cannot see the devices in /proc/partitions.

It's not meant to be - you'll only see the dm-N names here. That's
what udev refers to as the "kernel name" for the device. The entries
in /dev/mapper are aliases that are intended to be more
useful/meaningful to administrators. Internally they are the same device.

Q4: Does my setup described here look OK ?

See comments below.

The blades are running:

SLES 10 SP2 (x86-64)

From the MD3000i Dell Resource CD the following rpm was


No idea what this contains.

The following modules are running on my blades:

# lsmod | egrep 'rdac|mpp|multi' dm_rdac 24832 0 dm_multipath 42640 3 dm_rdac,dm_round_robin dm_mod
80528  26 dm_multipath

These are the modules needed for device-mapper multipathing.

mppVhba               145120  24 mppUpper              127436  1

These are the IBM MPP modules. You should not load these if using
device-mapper for multipathing. If you want to use MPP instead, refer
to the IBM documentation.

device { vendor DELL product "PERC|Universal" }

Shouldn't be required as you seem to have blacklisted these devices by
device node.

devices { device { vendor                  "DELL" product

These don't match the vendor/product details in the multipath output

You should check what the MD3000i reports in sysfs. E.g. for an EMC
clariion device I see:

# cat /sys/block/sdj/device/vendor
# cat /sys/block/sdj/device/model

And the corresponding entry in multipath.conf is:

               vendor                  "DGC"
               product                 "*"

I suspect from the output below that you want:

               vendor                  "DELL"
               product                 "DELL,MD Virtual Disk"

But check in sysfs.

multipaths { mulitpath { device { vendor DELL product MD3000i } } }

This block does not define anything.

After changing the multipath.conf I did:

multipath -F service multipathd restart

Now, if I've do:

# multipath -ll mpathe (36002219000918b4c00005b6c497e6632) dm-4
DELL,MD Virtual Disk [size=5.5T][features=0][hwhandler=0] \_
round-robin 0 [prio=1][active] \_ 9:0:0:1  sdi 8:128

Your settings are being ignored, probably because of a vendor/product


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]