[dm-devel] Multipath setup and rdac questions

Bryn M. Reeves bmr at redhat.com
Thu Feb 5 15:07:31 UTC 2009


Christian Røsnes wrote:
> Hi
> 
> I have got a couple of MD3000i with dual controllers, and two M600e
> blades connected two both of them.
> 
> I've setup multipath, and would appreciate if someone could tell me
> whether the setup described below looks correct, or if you'd do it
> any other way. The reason I'm asking, is that I cannot see any 
> reference to 'rdac hardware handler' when I do "multipath -ll".
> 
> However, when I change the module controller ownership one of the
> LUNs (while copying files to that particular LUN), the failover
> seems to work - the gilr copying continues after the controller
> switch.
> 
> I also see these messages in /var/log/messages when I change the
> module controller ownership:
> 
> 494 [RAIDarray.mpp]MD3000i_A:0:1:0 Cmnd-failed try alt ctrl 1.
> vcmnd SN 351494 pdev H2:C0:T0:L0 0x05/0x94/0x01 0x08000002
> mpp_status:1

Don't know where this is coming form. Do you have IBM's MPP
multipathing driver installed? If you are using device-mapper
multipath and multipath tools (the rest of your mail suggests this is
your intention) you should not load the MPP modules.

> Q1: What could be the reason that I'm not using the rdac hwhandler
> ? (see output from "multipath -ll" below)

Your vendor/product details don't seem to match what the array is
reporting.

> Q2: Is using rdac for this kind of setup preferred ?

Yes. It's required for proper failover if you're using device-mapper
multipathing.

> Q3: I've created a GPFS filesystem using the /dev/dm-0, /dev/dm-1,
> etc device. Is it OK to use the dm-x devices ? I see that there are
> other alternatives such as /dev/mapper/mpath<x>.

I always recommend using the /dev/mapper/* entries. These are
guaranteed to exist when the device is created (unlike the
udev-managed symlinks sometimes found in /dev/mpath) and are more
meaningful than the "dm-N" names which are really an internal
device-mapper name for the device.

> However, /dev/mapper/mpath<x> is not visible in /proc/partitions 
> (see below), but the dm-<x> devices are - and the GPFS filesystem
> complains it cannot see the devices in /proc/partitions.

It's not meant to be - you'll only see the dm-N names here. That's
what udev refers to as the "kernel name" for the device. The entries
in /dev/mapper are aliases that are intended to be more
useful/meaningful to administrators. Internally they are the same device.

> Q4: Does my setup described here look OK ?

See comments below.


> The blades are running:
> 
> SLES 10 SP2 (x86-64)
> 
>> From the MD3000i Dell Resource CD the following rpm was
>> installed:
> 
> linuxrdac-09.03.0C06.0030-1dkms

No idea what this contains.

> The following modules are running on my blades:
> 
> # lsmod | egrep 'rdac|mpp|multi' dm_rdac                24832  0 
> dm_multipath           42640  3  dm_rdac,dm_round_robin dm_mod
> 80528  26 dm_multipath

These are the modules needed for device-mapper multipathing.

> mppVhba               145120  24 mppUpper              127436  1
> mppVhba

These are the IBM MPP modules. You should not load these if using
device-mapper for multipathing. If you want to use MPP instead, refer
to the IBM documentation.

> device { vendor DELL product "PERC|Universal" }

Shouldn't be required as you seem to have blacklisted these devices by
device node.

> devices { device { vendor                  "DELL" product
> "MD3000i"

These don't match the vendor/product details in the multipath output
below.

You should check what the MD3000i reports in sysfs. E.g. for an EMC
clariion device I see:

# cat /sys/block/sdj/device/vendor
DGC
# cat /sys/block/sdj/device/model
RAID 5

And the corresponding entry in multipath.conf is:

                vendor                  "DGC"
                product                 "*"

I suspect from the output below that you want:

                vendor                  "DELL"
                product                 "DELL,MD Virtual Disk"

But check in sysfs.

> multipaths { mulitpath { device { vendor DELL product MD3000i } } }
> 
This block does not define anything.

> After changing the multipath.conf I did:
> 
> multipath -F service multipathd restart
> 
> Now, if I've do:
> 
> # multipath -ll mpathe (36002219000918b4c00005b6c497e6632) dm-4
> DELL,MD Virtual Disk [size=5.5T][features=0][hwhandler=0] \_
> round-robin 0 [prio=1][active] \_ 9:0:0:1  sdi 8:128
> [active][ready]

Your settings are being ignored, probably because of a vendor/product
mismatch.

Regards,
Bryn.





More information about the dm-devel mailing list