[linux-lvm] device-mapper may have read performance issue

Thu Oct 9 09:53:08 UTC 2008

Hi all,

On Wed, 8 Oct 2008, thomas62186218 at aol.com wrote:

> Ben,
>
> I have seen this same issue as well. I have created an md device capable of 
> 425MB/sec using the hdparm -t command, yet an LVM volume fully comprising 
> this md device only got about 150MB/sec. I am not sure what the issue is. I 
> am running Ubuntu Hardy 804 server edition, 64-bit.
>
> -Thomas

I have fixed this kind of problem by tweaking the readahead of the LVM 
volume using 'blockdev --setra' and/or 'blockdev --setfra'.

I originally found this tip here: 
http://www.beowulf.org/archive/2007-May/018359.html

Good luck,
Peter.

>
>
> -----Original Message-----
> From: Ben Huang <ben_devel at yahoo.cn>
> To: linux-lvm at redhat.com
> Sent: Wed, 8 Oct 2008 12:51 am
> Subject: [linux-lvm] device-mapper may have read performance issue
>
>
>
>
>
>
>
>
>
> HI all,
>      I found a LVM+dm read performance issue on my storage server. On my 
> system, I have a 12+0 RAID5 md, which was created by the following command
>      mdadm -C /dev/md0 -l5 -n12 /dev/sd{a,b,l,d,e,f,g,h,i,j,k,o} 
> --assume-clean --metadata=1
>
>    My first test was
>    dd if=/dev/md0 of=/dev/null bs=1M
>    And I got around 780MB/s sequence read performance
>
> vmstat 1
>
> procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
>  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
>  2  0      0  31272 3369856 258728    0    0
> 794680     0 3223 4120  0 35 61  3
>  1  0      0  24980 3371104 258664A
> 0   0    0 788484     0 3271 3909  0 31 67  2
>  1  0      0  35208 3363600 258760    0    0 792572     0 3288 4056  0 34 65  
> 1
>  0  1      0  22304 3380164 258560    0    0 753668     0 2875 3775  0 29 70  
> 1
>  1  0      0  41272 3361048 258660    0    0 770048     0 3124 3977  0 32 66  
> 2
>  1  0      0  32636 3366976 258536    0    0 759485     0 2920 3823  0 33 64 
> 3
>  1  0      0  21796 3376132 258632    0    0 776184     0 3053 3995  0 33 66  
> 2
>  1  0      0  28612 3373608 258872    0    0 758148     0 3050 3907  0 30 68  
> 3
>  3  0      0  35768 3366948 258504    0    0 778240     0 3154 3873  0 32 66  
> 1
>  1  0      0  25588 3377328 258864    0    0 737296     0 2913 3262  0 30 68  
> 2
>  1  0      0  31264 3370300 258828    0 =C
> 2  0 792832     0 3339 2934  0 34 63  2
>  0  1      0  21532 3380072
> 258708    0    0 755512     0 3067 2986  0 29 68  3
>
>     my second test is,
>     pvcreate /dev/md0
>     vgcreate DG5 /dev/md0
>     lvcreate -L 200G -n vd1 DG5
>     dd if=/dev/mapper/DG5-vd1 of=/dev/null bs=1M
>     The sequence read performance dropped to 340MB/s
>
> vmstat 1
>
> procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
>  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id
> wa
>  1  0      0  76440 3327728 258884    0    0 344612     0 3319 7564  0 19 65 
> 16
>  0  1      0  29920 3374372 258636    0    0 345088     0 3290 7620  0 21 67 
> 12
>  1  0      0  32608 3371800 258856    0    0 334464     0 3154 7513  0 19 67 
> 14
>  0  1      0  32340 3371800 258784    0    0 341176     0 3289 7538  0 17 69 
> 14
>  0  1      0  31812 3372424258516    0    0 337664     0 3148 7443  0 19 68 
> 13
>  2  0      0  23964 3380052
> 258372    0    0 337792     0 3227 7656  0 19 67 14
>  1  0      0  38496 3365564 258696    0    0 344564     0 3297 7690  0 20 66 
> 13
>  2  0      0  24120 3379592 258584    0    0 339468     0 3191 7448  0 19 68 
> 13
>  1  1      0  25072 3378380 258640    0    0 338360     0 3153 7471  0 17 73 
> 11
>  1  1      0  61488 3341904 258808    0    0 340992     0 3257 7721  0 21 64 
> 15
>  0  1      0  51504 3352048 258568    0    0 342528     0 3270 7648  0
> 20 65 15
>  1  0      0  89064 3314560 258508    0    0 340224     0 3222 7764  0 20 68 
> 13
>  1  0      0  32008 3371572 258740    0    0 337716     0 3218 7302  0 21 63 
> 17
>  0  1      0  80608 3322828 258972    0    0 335364   A
> 0 0 3157 7661  0 18 62 21
>
>
> root:~# uname -a
> Linux ustor 2.6.26.2 #1 SMP PREEMPT Sun Sep 28 11:58:17 CST 2008 x86_64 
> x86_64 x86_64 GNU/Linux
>
> root:~# lvm version
>   LVM version:     2.02.39 (2008-06-27)
>   Library version: 1.02.27 (2008-06-25)
>   Driver version:  4.13.0
> (The lvm snapshot merging patches has been applied)
>
>
> I used the fowllowing scripts to monitor the
> /proc/diskstats,
>
> root:~ # while true; do clear; cat /proc/diskstats |grep -E "sd|md|dm" |awk 
> root:~ '{printf "[%4s] %10s %10s %10s %10s\t%10s %10s %10s %10s\t%10s %10s 
> root:~ %10s\n",$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14}'; sleep 1; done
>
> I found that dm may hold too many IOs in its queue
>
> Has anyone met with the same issues?
> Sorry for my poor english
>
>
>
>   
>
>
>
>
>
>
>
> ------------------------------------------------------------
> 雅虎邮箱，您的终生邮箱！
>
>
>
>
>
>
>
>
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm at redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>
>
>
>
>
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm at redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>

--
Peter Keller                                     Tel.: +44 (0)1223 353033
Global Phasing Ltd.,                             Fax.: +44 (0)1223 366889
Sheraton House,
Castle Park,
Cambridge CB3 0AX
United Kingdom