[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: EXT3 and large directories




(please refrain from sendin HTML mails)

On Wed, 4 Oct 2006, Matt Dodson wrote:
I have an ext3 filesystem that has several directories and each
directory gets a large number of files inserted and then deleted over
time.

Can you specify "large number"? What do "ls large-directory | wc -l" say?

The filesystem is basically used as a temp store before files are
processed. The issue is over time the directory scans get extremely slow
even if the directories are empty. I have noticed the directories can
range in size from 4k - 100M even when they are empty.

proably deleted-but-still-open files. When lsof(8) is installed, you can find out with: "lsof -ln | grep large-but-empty-directory"

Can you specify "slow" as well? You also might want strace(1) an "ls" on your large directory to see what is taking so long.

Is there a way to fix this without recreating the directories or
bringing the filesystem offline?

You have enabled htree (dir_index) already:

Filesystem features:      has_journal resize_inode dir_index filetype
needs_recovery sparse_super large_file

If you've enabled dir_index after the directories have been created, you might want to "e2fsck -D" (see the manpage for details) the filesystem. For partitions with temprary files you could play with "noatime","async" and "data" mount-options (please read the manpage, really!).
Which kernel do you use? Which arch?

C.
--
BOFH excuse #83:

Support staff hung over, send aspirin and come back LATER.


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]