[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [dm-devel] understanding of multipathing and speed



Hi,
On Wed, 2010-07-07 at 23:01 +0200, Bart Coninckx wrote:
> On Wednesday 07 July 2010 19:18:48 Bart Coninckx wrote:
> > On Tuesday 06 July 2010 06:16:55 Bart Coninckx wrote:
> > > On Monday 05 July 2010 20:58:30 Christophe Varoqui wrote:
> > > > On lun., 2010-07-05 at 20:37 +0200, Bart Coninckx wrote:
> > > > > Hi,
> > > > >
> > > > > I would like to run my ideas by this list about multipathing and the
> > > > > results as far as storage speed is concerned.
> > > > >
> > > > > I'm using multipathing to two iSCSI targets pointing to the same
> > > > > storage. It was my understanding that this provides for network path
> > > > > redundancy (and it does, I tested this) but also for added speed.
> > > > > I did some tests with Bonnie++ however while both paths were active
> > > > > and one path was down and the results are basically the same.
> > > > >
> > > > > Am I assuming wrong things? Or have I configured things wrong?
> > > >
> > > > can you also include a 'multipath -l' output and sketch the
> > > > hba/switch/controller physical connections ?
> > > >
> > > > thanks,
> > >
> > > Sure,
> > >
> > > xen3:~ # multipath -l
> > > lx03 (1494554000000000000000000010000000000000002000000) dm-3
> > >  IET,VIRTUAL-DISK [size=10G][features=1 queue_if_no_path][hwhandler=0]
> > > \_ round-robin 0 [prio=-2][active]
> > >  \_ 2:0:0:0 sdc 8:32  [active][undef]
> > >  \_ 1:0:0:0 sdb 8:16  [active][undef]
> > > ws033 (1494554000000000000000000010000000100000002000000) dm-2
> > > IET,VIRTUAL- DISK
> > > [size=15G][features=1 queue_if_no_path][hwhandler=0]
> > > \_ round-robin 0 [prio=-2][active]
> > >  \_ 2:0:0:1 sde 8:64  [active][undef]
> > >  \_ 1:0:0:1 sdd 8:48  [active][undef]
> > > ms01 (1494554000000000000000000010000000200000002000000) dm-1
> > >  IET,VIRTUAL-DISK [size=40G][features=1 queue_if_no_path][hwhandler=0]
> > > \_ round-robin 0 [prio=-2][active]
> > >  \_ 1:0:0:2 sdf 8:80  [active][undef]
> > >  \_ 2:0:0:2 sdg 8:96  [active][undef]
> > >
> > > I have two Gigabit NICs in this server each running over a separate
> > > switch to a separate gigabit NIC with a unique IP address on the storage
> > > IET iSCSI target.
> > >
> > > Is this sufficient info?
> > >
> > > Thx,
> > >
> > > Bart
> > >
> > > --
> > > dm-devel mailing list
> > > dm-devel redhat com
> > > https://www.redhat.com/mailman/listinfo/dm-devel
> > 
> > Hi all,
> > 
> > to show my point, these are the results of running bonnie++ locally on the
> > storage - the values I look at are Block values in K/sec in both sequential
> > output (writing) and sequential input (reading):
> > 
> > Version 1.03e       ------Sequential Output------ --Sequential Input- --
> > Random-
> >                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --
> > Seeks--
> > Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec
> > %CP
> > iscsi3           8G 69351  96 116112  32 41128  10 57874  82 107721  16
> >  418.2 0
> >                     ------Sequential Create------ --------Random
> > Create--------
> >                     -Create-- --Read--- -Delete-- -Create-- --Read--- -
> > Delete--
> >               files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec
> > %CP
> >                  16  4533  99 +++++ +++ +++++ +++  4395  99 +++++ +++ 17122
> > 99
> > iscsi3,8G,69351,96,116112,32,41128,10,57874,82,107721,16,418.2,0,16,4533,99
> > , +++++,+++,+++++,+++,4395,99,+++++,+++,17122,99
> > 
> > 
> > 
> > So were are hitting roughly 110 MB/sec locally on the storage server.
> > 
> > Now these are the results do doing the same over multipath with two paths
> > enabled:
> > 
> > Version 1.03e       ------Sequential Output------ --Sequential Input- --
> > Random-
> >                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --
> > Seeks--
> > Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec
> > %CP
> > xen3             8G 63953  92 100525  26 26885   2 41957  55 68184   2
> >  357.9 0
> >                     ------Sequential Create------ --------Random
> > Create--------
> >                     -Create-- --Read--- -Delete-- -Create-- --Read--- -
> > Delete--
> >               files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec
> > %CP
> >                  16  5326  98 +++++ +++ +++++ +++  5333  97 +++++ +++ 17179
> > 100
> > xen3,8G,63953,92,100525,26,26885,2,41957,55,68184,2,357.9,0,16,5326,98,++++
> > +, +++,+++++,+++,5333,97,+++++,+++,17179,100
> > 
> > You can see we hit somewhat less, probably due to TCP overhead (though this
> > should cut things with 30%). Now the same with one path down:
> > 
> > Version 1.03e       ------Sequential Output------ --Sequential Input- --
> > Random-
> >                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --
> > Seeks--
> > Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec
> > %CP
> > xen3             8G 33214  46 113811  29 27917   1 44474  58 68812   2
> >  362.8 0
> >                     ------Sequential Create------ --------Random
> > Create--------
> >                     -Create-- --Read--- -Delete-- -Create-- --Read--- -
> > Delete--
> >               files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec
> > %CP
> >                  16  5294  98 +++++ +++ +++++ +++  5337  97 +++++ +++ 17183
> > 99
> > xen3,8G,33214,46,113811,29,27917,1,44474,58,68812,2,362.8,0,16,5294,98,++++
> > +, +++,+++++,+++,5337,97,+++++,+++,17183,99
> > 
> > As you can see, roughly the same K/sec for both output and input. Actually
> > writing is even faster with one path down!
> > Can anyone make sense of these values?
> > 
> > thx!
> > 
> > B.
> > 
> > 
> > --
> > dm-devel mailing list
> > dm-devel redhat com
> > https://www.redhat.com/mailman/listinfo/dm-devel
> > 
> 
> Maybe adding this one while doing a test with both paths active during "off 
> hours", so no other intrusive factors: 
> 
> Version 1.03e       ------Sequential Output------ --Sequential Input- --
> Random-
>                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --
> Seeks--
> Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec 
> %CP
> xen3             8G 66510  93 80841  21 26821   1 45368  58 72095   2 361.2   
> 0
>                     ------Sequential Create------ --------Random 
> Create--------
>                     -Create-- --Read--- -Delete-- -Create-- --Read--- -
> Delete--
>               files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec 
> %CP
>                  16  5295  98 +++++ +++ +++++ +++  5318  98 +++++ +++ 17089 
> 100
> xen3,8G,66510,93,80841,21,26821,1,45368,58,72095,2,361.2,0,16,5295,98,+++++,
> +++,+++++,+++,5318,98,+++++,+++,17089,100
> 
> 
> it show that the speed is exactly 70% of the speed when doing tests locally. 
> So this might be the iSCSI TCP overhead. 
> 
> Should the speed of two round robin paths not compensate for this loss? Or is 
> my local storage just to slow to have multipath having any benefit speed wise?
> 
To calculate your theoretical TCP throughput, a simple formula can be
applied : TP= TCP Window size / RTT 

In addition to this generally, you can take advantage of path
load-balancing IO's to capable to multiplex IO's devices , such as SAN
arrays with cache frontend.
Trying to load-balance on a single physical device, won't be, IMHO, of
any help except for pure failover purposes.
   
> 
> thx!
> 
> B.
> 
> --
> dm-devel mailing list
> dm-devel redhat com
> https://www.redhat.com/mailman/listinfo/dm-devel



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]