[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [linux-lvm] device-mapper may have read performance issue



On Thu, 9 Oct 2008, Peter Keller wrote:

Hi all,

On Wed, 8 Oct 2008, thomas62186218 aol com wrote:

 Ben,

 I have seen this same issue as well. I have created an md device capable
 of 425MB/sec using the hdparm -t command, yet an LVM volume fully
 comprising this md device only got about 150MB/sec. I am not sure what the
 issue is. I am running Ubuntu Hardy 804 server edition, 64-bit.

 -Thomas

I have fixed this kind of problem by tweaking the readahead of the LVM volume using 'blockdev --setra' and/or 'blockdev --setfra'.

I originally found this tip here: http://www.beowulf.org/archive/2007-May/018359.html

Good luck,
Peter.

I should also have said: don't forget to use 'blockdev --flushbufs' between tests.

Regards,
Peter.




 -----Original Message-----
 From: Ben Huang <ben_devel yahoo cn>
 To: linux-lvm redhat com
 Sent: Wed, 8 Oct 2008 12:51 am
 Subject: [linux-lvm] device-mapper may have read performance issue









 HI all,
      I found a LVM+dm read performance issue on my storage server. On my
 system, I have a 12+0 RAID5 md, which was created by the following command
      mdadm -C /dev/md0 -l5 -n12 /dev/sd{a,b,l,d,e,f,g,h,i,j,k,o}
 --assume-clean --metadata=1

    My first test was
    dd if=/dev/md0 of=/dev/null bs=1M
    And I got around 780MB/s sequence read performance

 vmstat 1

 procs -----------memory---------- ---swap-- -----io---- -system--
 ----cpu----
  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id
 wa
  2  0      0  31272 3369856 258728    0    0
 794680     0 3223 4120  0 35 61  3
  1  0      0  24980 3371104 258664A
 0   0    0 788484     0 3271 3909  0 31 67  2
  1  0      0  35208 3363600 258760    0    0 792572     0 3288 4056  0 34
 65  1
  0  1      0  22304 3380164 258560    0    0 753668     0 2875 3775  0 29
 70  1
  1  0      0  41272 3361048 258660    0    0 770048     0 3124 3977  0 32
 66  2
  1  0      0  32636 3366976 258536    0    0 759485     0 2920 3823  0 33
 64 
 3
  1  0      0  21796 3376132 258632    0    0 776184     0 3053 3995  0 33
 66  2
  1  0      0  28612 3373608 258872    0    0 758148     0 3050 3907  0 30
 68  3
  3  0      0  35768 3366948 258504    0    0 778240     0 3154 3873  0 32
 66  1
  1  0      0  25588 3377328 258864    0    0 737296     0 2913 3262  0 30
 68  2
  1  0      0  31264 3370300 258828    0 =C
 2  0 792832     0 3339 2934  0 34 63  2
  0  1      0  21532 3380072
 258708    0    0 755512     0 3067 2986  0 29 68  3

     my second test is,
     pvcreate /dev/md0
     vgcreate DG5 /dev/md0
     lvcreate -L 200G -n vd1 DG5
     dd if=/dev/mapper/DG5-vd1 of=/dev/null bs=1M
     The sequence read performance dropped to 340MB/s

 vmstat 1

 procs -----------memory---------- ---swap-- -----io---- -system--
 ----cpu----
  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id
 wa
  1  0      0  76440 3327728 258884    0    0 344612     0 3319 7564  0 19
 65 16
  0  1      0  29920 3374372 258636    0    0 345088     0 3290 7620  0 21
 67 12
  1  0      0  32608 3371800 258856    0    0 334464     0 3154 7513  0 19
 67 14
  0  1      0  32340 3371800 258784    0    0 341176     0 3289 7538  0 17
 69 14
  0  1      0  31812 3372424258516    0    0 337664     0 3148 7443  0 19
 68 13
  2  0      0  23964 3380052
 258372    0    0 337792     0 3227 7656  0 19 67 14
  1  0      0  38496 3365564 258696    0    0 344564     0 3297 7690  0 20
 66 13
  2  0      0  24120 3379592 258584    0    0 339468     0 3191 7448  0 19
 68 13
  1  1      0  25072 3378380 258640    0    0 338360     0 3153 7471  0 17
 73 11
  1  1      0  61488 3341904 258808    0    0 340992     0 3257 7721  0 21
 64 15
  0  1      0  51504 3352048 258568    0    0 342528     0 3270 7648  0
 20 65 15
  1  0      0  89064 3314560 258508    0    0 340224     0 3222 7764  0 20
 68 13
  1  0      0  32008 3371572 258740    0    0 337716     0 3218 7302  0 21
 63 17
  0  1      0  80608 3322828 258972    0    0 335364   A
 0 0 3157 7661  0 18 62 21


 root:~# uname -a
 Linux ustor 2.6.26.2 #1 SMP PREEMPT Sun Sep 28 11:58:17 CST 2008 x86_64
 x86_64 x86_64 GNU/Linux

 root:~# lvm version
   LVM version:     2.02.39 (2008-06-27)
   Library version: 1.02.27 (2008-06-25)
   Driver version:  4.13.0
 (The lvm snapshot merging patches has been applied)


 I used the fowllowing scripts to monitor the
 /proc/diskstats,

root:~ # while true; do clear; cat /proc/diskstats |grep -E "sd|md|dm" root:~ |awk '{printf "[%4s] %10s %10s %10s %10s\t%10s %10s %10s %10s\t%10s root:~ %10s %10s\n",$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14}'; sleep 1; root:~ done

 I found that dm may hold too many IOs in its queue

 Has anyone met with the same issues?
 Sorry for my poor english



   







 ------------------------------------------------------------
 雅虎邮箱,您的终生邮箱!









 _______________________________________________
 linux-lvm mailing list
 linux-lvm redhat com
 https://www.redhat.com/mailman/listinfo/linux-lvm
 read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/






 _______________________________________________
 linux-lvm mailing list
 linux-lvm redhat com
 https://www.redhat.com/mailman/listinfo/linux-lvm
 read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/


--
Peter Keller                                     Tel.: +44 (0)1223 353033
Global Phasing Ltd.,                             Fax.: +44 (0)1223 366889
Sheraton House,
Castle Park,
Cambridge CB3 0AX
United Kingdom


--
Peter Keller                                     Tel.: +44 (0)1223 353033
Global Phasing Ltd.,                             Fax.: +44 (0)1223 366889
Sheraton House,
Castle Park,
Cambridge CB3 0AX
United Kingdom
_______________________________________________
linux-lvm mailing list
linux-lvm redhat com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]