[dm-devel] understanding of multipathing and speed
Bart Coninckx
bart.coninckx at telenet.be
Wed Jul 7 21:01:47 UTC 2010
On Wednesday 07 July 2010 19:18:48 Bart Coninckx wrote:
> On Tuesday 06 July 2010 06:16:55 Bart Coninckx wrote:
> > On Monday 05 July 2010 20:58:30 Christophe Varoqui wrote:
> > > On lun., 2010-07-05 at 20:37 +0200, Bart Coninckx wrote:
> > > > Hi,
> > > >
> > > > I would like to run my ideas by this list about multipathing and the
> > > > results as far as storage speed is concerned.
> > > >
> > > > I'm using multipathing to two iSCSI targets pointing to the same
> > > > storage. It was my understanding that this provides for network path
> > > > redundancy (and it does, I tested this) but also for added speed.
> > > > I did some tests with Bonnie++ however while both paths were active
> > > > and one path was down and the results are basically the same.
> > > >
> > > > Am I assuming wrong things? Or have I configured things wrong?
> > >
> > > can you also include a 'multipath -l' output and sketch the
> > > hba/switch/controller physical connections ?
> > >
> > > thanks,
> >
> > Sure,
> >
> > xen3:~ # multipath -l
> > lx03 (1494554000000000000000000010000000000000002000000) dm-3
> > IET,VIRTUAL-DISK [size=10G][features=1 queue_if_no_path][hwhandler=0]
> > \_ round-robin 0 [prio=-2][active]
> > \_ 2:0:0:0 sdc 8:32 [active][undef]
> > \_ 1:0:0:0 sdb 8:16 [active][undef]
> > ws033 (1494554000000000000000000010000000100000002000000) dm-2
> > IET,VIRTUAL- DISK
> > [size=15G][features=1 queue_if_no_path][hwhandler=0]
> > \_ round-robin 0 [prio=-2][active]
> > \_ 2:0:0:1 sde 8:64 [active][undef]
> > \_ 1:0:0:1 sdd 8:48 [active][undef]
> > ms01 (1494554000000000000000000010000000200000002000000) dm-1
> > IET,VIRTUAL-DISK [size=40G][features=1 queue_if_no_path][hwhandler=0]
> > \_ round-robin 0 [prio=-2][active]
> > \_ 1:0:0:2 sdf 8:80 [active][undef]
> > \_ 2:0:0:2 sdg 8:96 [active][undef]
> >
> > I have two Gigabit NICs in this server each running over a separate
> > switch to a separate gigabit NIC with a unique IP address on the storage
> > IET iSCSI target.
> >
> > Is this sufficient info?
> >
> > Thx,
> >
> > Bart
> >
> > --
> > dm-devel mailing list
> > dm-devel at redhat.com
> > https://www.redhat.com/mailman/listinfo/dm-devel
>
> Hi all,
>
> to show my point, these are the results of running bonnie++ locally on the
> storage - the values I look at are Block values in K/sec in both sequential
> output (writing) and sequential input (reading):
>
> Version 1.03e ------Sequential Output------ --Sequential Input- --
> Random-
> -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --
> Seeks--
> Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
> %CP
> iscsi3 8G 69351 96 116112 32 41128 10 57874 82 107721 16
> 418.2 0
> ------Sequential Create------ --------Random
> Create--------
> -Create-- --Read--- -Delete-- -Create-- --Read--- -
> Delete--
> files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
> %CP
> 16 4533 99 +++++ +++ +++++ +++ 4395 99 +++++ +++ 17122
> 99
> iscsi3,8G,69351,96,116112,32,41128,10,57874,82,107721,16,418.2,0,16,4533,99
> , +++++,+++,+++++,+++,4395,99,+++++,+++,17122,99
>
>
>
> So were are hitting roughly 110 MB/sec locally on the storage server.
>
> Now these are the results do doing the same over multipath with two paths
> enabled:
>
> Version 1.03e ------Sequential Output------ --Sequential Input- --
> Random-
> -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --
> Seeks--
> Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
> %CP
> xen3 8G 63953 92 100525 26 26885 2 41957 55 68184 2
> 357.9 0
> ------Sequential Create------ --------Random
> Create--------
> -Create-- --Read--- -Delete-- -Create-- --Read--- -
> Delete--
> files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
> %CP
> 16 5326 98 +++++ +++ +++++ +++ 5333 97 +++++ +++ 17179
> 100
> xen3,8G,63953,92,100525,26,26885,2,41957,55,68184,2,357.9,0,16,5326,98,++++
> +, +++,+++++,+++,5333,97,+++++,+++,17179,100
>
> You can see we hit somewhat less, probably due to TCP overhead (though this
> should cut things with 30%). Now the same with one path down:
>
> Version 1.03e ------Sequential Output------ --Sequential Input- --
> Random-
> -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --
> Seeks--
> Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
> %CP
> xen3 8G 33214 46 113811 29 27917 1 44474 58 68812 2
> 362.8 0
> ------Sequential Create------ --------Random
> Create--------
> -Create-- --Read--- -Delete-- -Create-- --Read--- -
> Delete--
> files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
> %CP
> 16 5294 98 +++++ +++ +++++ +++ 5337 97 +++++ +++ 17183
> 99
> xen3,8G,33214,46,113811,29,27917,1,44474,58,68812,2,362.8,0,16,5294,98,++++
> +, +++,+++++,+++,5337,97,+++++,+++,17183,99
>
> As you can see, roughly the same K/sec for both output and input. Actually
> writing is even faster with one path down!
> Can anyone make sense of these values?
>
> thx!
>
> B.
>
>
> --
> dm-devel mailing list
> dm-devel at redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
>
Maybe adding this one while doing a test with both paths active during "off
hours", so no other intrusive factors:
Version 1.03e ------Sequential Output------ --Sequential Input- --
Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --
Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
xen3 8G 66510 93 80841 21 26821 1 45368 58 72095 2 361.2
0
------Sequential Create------ --------Random
Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -
Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
16 5295 98 +++++ +++ +++++ +++ 5318 98 +++++ +++ 17089
100
xen3,8G,66510,93,80841,21,26821,1,45368,58,72095,2,361.2,0,16,5295,98,+++++,
+++,+++++,+++,5318,98,+++++,+++,17089,100
it show that the speed is exactly 70% of the speed when doing tests locally.
So this might be the iSCSI TCP overhead.
Should the speed of two round robin paths not compensate for this loss? Or is
my local storage just to slow to have multipath having any benefit speed wise?
thx!
B.
More information about the dm-devel
mailing list