[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [linux-lvm] LVM causing IO contention or slowdown?



Austin Gonyou wrote:

As a follow up to this I've done some more testing, and will test the
rest of this weekend, using AIM db benchmark.


What I've found is that when mounting with logbufs=8 on my Quad Xeon 2MB
cache 6450 with 8 Ultra-2 Drives, 4 RAID0 volumes, two of 3 disks each
and two of 1 disk each.


The larger Volumes, the more "costly" volumes, when mounted using
logbufs=8, outperformed the same volume addressed when not under LVM
control. Not only did it out perform itself, but either WITH or WITHOUT
LVM controlling the volume, I nearly doubled my throughput to those
drives.


Here is what I'm talking about:

#With LVM management
Throughput 109.715 MB/sec (NB=137.144 MB/sec  1097.15 MBit/sec)  200
procs

#Without LVM management
Throughput 100.803 MB/sec (NB=126.004 MB/sec  1008.03 MBit/sec)  200
procs


This only happened once though and I'm not sure exactly why. Still, the max I could get out of it for the same test more than once with LVM enabled was around 80-85 MB/s. Still a HUGE improvement over 43/44 MB/s.


Off topic for the lvm list, but....


The logbufs=8 parameter basically means you have 8 buffers capable of pushing
transactions out to disk. If you have lots of threads going at once in xfs then the
transactions tend to get throttled waiting for a buffer to do a log write into, so
adding more is good.


There is a perl script in the cmd/xfsmisc directory called xfs_stats.pl if you
run this it will format the output of the xfs /proc kernel statistics. You will
see a parameter called xs_log_noiclogs near the bottom of the first column.
This number means a transaction finished, but then had to wait for a log
buffer before it could hand off its data.


The downside of increasing the number of logbuffers is the amount of data
which can be lost after a crash (i.e. how many ops in the filesystem are
effectively undone by recovery). However,  looking at stats on my box,
each log write contains about 5 transactions, so you can never loose
much.

We have found adding more than 8 usually does not help, making them
bigger would be a different story, but that is actually a very non-trivial
change.

Steve





[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]