[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: ext3 performance issue with a Berkeley db application



Matthias Andree <matthias andree gmx de> wrote:
>
> On Sun, 02 Feb 2003, Andrew Morton wrote:
> 
> > > Here's the ext3 part.  On another machine I got the following times to
> > > build that list of 530,000 tokens, starting by creating empty db files
> > > on a partition mounted as:
> > 
> > How large are the generated output files?
> 
> Some ten MB, like 30.

hum.

> The access pattern changes a lot, because the data base is dumped in
> traversal order which makes reinserting them into a fresh tree have MUCH
> better data locality. Most writes are then in sequential order in
> respect to the file offset, with some excursions to offset #0 and #4096
> (pages #0 and #1), as you can see on

hum.

So you have an operation which takes 30 seconds and generates 30 megabytes of
dirty data.  My wild guess would be that the first five seconds' worth of
data are splattered all over the disk, and that the holes in that write
pattern are filled in later in the run.

So if a filesystem happens to decide to write the data after the first five
seconds, there is little request merging.  But if the filesystem were to wait
the whole thirty seconds then voila - all the holes are filled in and there's
a lot of request merging.

Well.  It's a theory.  If you mount your data=ordered fs with `-o commit=60',
what happens?

> Are you aware of a module that applies to recent kernel versions and
> that traces block numbers of ll_rw_block()? It might turn up some useful
> information -- then we'd easier know the ext3 "output" to the hard disk;
> we already know the "input" from the application at the syscall level.

Not really.  I stick printk's in the kernel, give it a big printk buffer and
kill syslog.

> 
> BTW: what's the status of the dirsync patches in respect to 2.4.21-pre?
> Is further testing needed or just a "ping" to get them merged? Or
> does Marcelo not want the patch?

I just haven't got onto it, sorry.





[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]