ext3 with maildir++ = huge disk latency and high load

[Andrey]

Ok. Here are bonnie results on fresh installed Debian with 200GB FREE 
ext3 /home partitition (4 disks in RAID5 on HP Proliant DL380 G4 server):

Version  1.96  ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- 
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP 
/sec %CP
debian           2G   243  97 22555  10  8794   2  1810  97 120444  11 
317.0   5
Latency               135ms     967ms     723ms   26526us   13143us 
586ms

Latency is also very bad according results. What is the reason? Hardware 
or ext3 itseld? Will try with xfs an ext4 and compare then.


26.09.2011 01:21, Ted Ts'o пишет:
> On Sun, Sep 25, 2011 at 12:16:12AM -0600, Andreas Dilger wrote:
>>
>> It would be possible to do something like this in the ext4 readdir
>> code to do dirent readahead, sort, and then prefetch the inodes
>> in order (partially or completely, depending on the directory size),
>> but as yet we aren't working on anything at the ext4 level.
>
> What we have in ext4 right now is if we need to do disk i/o to read
> from the inode table, we will read in adjacent blocks from the inode
> table, on the theory that the effort needed to read in 32k versus 4k
> is pretty much the same.  So if the inodes were allocated all at the
> same time, they will be sequentially ordered, and so the inode table
> readahead should help quite a lot.
>
> I'll note that with really large maildirs, especially on a mail server
> with many other maildirs, over time the inodes for each individual
> file will get scattered all over the place, and so pretty much any
> scheme that uses a inode table separate from the blocks where the
> directory entries are stored is going to get hammered by this use
> case.
>
> Ultimately, the best way to solve this problem is a more intelligent
> application that caches the contents of the key headers in a database,
> so you don't need to scan the contents of the entire Maildir when
> doing common IMAP operations.
>
> 						- Ted
>
>