[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Journal Usage



On Tue, Nov 13, 2001 at 04:43:38PM +0000, Stephen C. Tweedie wrote:
> Hi,
> 
> On Thu, Nov 08, 2001 at 01:46:57AM -0800, Mike Fedyk wrote:
> > > 
> > > You can make the journal flush interval longer, so that it will flush only
> > 
> > And how would I do this safely?  Is it just modifying a define somewhere?
> 
> It _ought_ to be that easy, but it's not, yet!
> 
> You currently need to modify the line
> 
> 	journal->j_commit_interval = (HZ * 5);
> 
> in linux/fs/jbd/journal.c.  I'll add this as a per-fs tunable.
>

Yep, but it's still just a one liner patch just like a #define would be...
Though the proc interface will be nice. :)

> > > when the journal is getting full, rather than after a fixed amount of time.
> > > This may not be ideal either, because if you are forced to flush because
> > > the journal is getting full it will likely be just at a time when you want
> > > to write into it.
> > 
> > I'd propose something more gradual besides "oops, the journal is full, let's
> > empty it now..."
> 
> The advantage to leaving it as long as possible is that you minimise
> the amount that needs to be flushed.  If you flush too soon you risk
> doing quite a bit of unnecessary IO.
> 

Actually, it looks like this will be needed anyway to deal with large
ammounts of IO to the journal.  There already seems to be a patch that does
this out there somewhere a while back (someone mentioned here or on
ext2-devel).

Personally, I've set my journal size to the max (400MB on 4k blocks) and I'd
like to see more of it used.  Right now, I can't even tell how much is being
used now, but from what I can see, it is comitting from the journal to the
FS with much less than 100MB written out.

This is a subjective test though, just watching mutt write a large mail
spool, and hearing the disk start seeking all over the place (ie, not
journal writes) after less than 20% of 112MB has been written.

To be specific, I'm just suggesting there should be more points where the
journal changes its writing behaviour.  

Low percentage used (20%<)?  Don't write out if there are transactions writing
aggressively to the journal. 

Med percentage used (20-40%)?  Start writing out slowly...

Med-high percentage used(40-70)?  Write out more aggressively.

High percentage used (70-85%)?  Write out as hard as possible without
blocking.

Overflow (<85%)?  Write out *hard*, while blocking.

Once the percentage of journal usage moves into another category, the write
mode will change accordingly.  ie, if we hit overflow mode, it will stop
blocking when journal usage goes back down to 84%...

Now, I'm thinking of rather large journals in data=journal mode.  In smaller
journals we may hit overflow mode more often than with larger journals.
Though, IIRC, there are only about two marks that change the journal write
out mode now...

One question.  If I write something that goes completely into the journal
(large 400MB journal, data=journal) the write is still in memory.  If there
is memory pressure, that'll cause the completed transactions to be written
out to the FS to free some memory...  Does that mean that I need 400MB of
memory that stays cached to use my entire 400MB journal?

Mike





[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]