[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: ext3 performance issue with a Berkeley db application

Andrew Morton <akpm digeo com> writes:

> ext3 is making it write much _more_ data.  Due to the 5-second commit,
> the application's redirtying and an ext3 gremlin.

So this means we write more data because ext3 commits quicker than the
other file systems I tested and has less chance to kill old write
requests that are overwritten because they were already flushed to disk?
Then we really won't have much choice except quenching our overwrites as
far as reasonable and try to improve locality of our operations.

The part that still interests me is: What I still don't understand is
that ext3 on SCSI schedules some writes in 5 s intervals with 4 idle
seconds at most times; then some bursts of 10,000 or 30,000 blocks
occasionally and writes a consistent 1,000 to 2,500 blocks/s on fsync --
IDE in contrast has a constant write rate of 700 blocks/s without these
4 seconds at 0 blocks. Why is my SCSI layer faster than Greg's although
my drive is slower?

I did this: run my simbf changed to to 2 million writes to 20,000
distinct pages, i. e. 80,000 kByte output file. With the big data and
aborted after 45 s. I started vmstat 1 >/dev/shm/e3a before starting
simbf and killed it after 50 s.

I extracted the io bo column from vmstat output with this line:

awk '{ if ($1+0 > 0) { print $11; } }' /dev/shm/e3a | head -n 50

for ATA and again for SCSI (/dev/shm/e3s) and plotted the beast.

Please look at http://mandree.home.pages.de/bogofilter/vmstat-1.png -- I
hope this is clearer and tells more than a thousand words.

So again: is it worth trying with a vanilla, Marcelo, ... kernel? Is
there anything I can tune, elvtune, mount options, ext3fs options that
might differ between this file systems so I can eliminate perturbations?

I have BK, CVS and all that, so I can test virtually any kernel if that
is of any help.

Matthias Andree

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]