Tuning for large number of directory entries?

Huntress Gary B NPRI HuntressGB at Npt.NUWC.Navy.Mil
Sat Apr 1 14:53:11 UTC 2006



I have been running a public MySQL server for over 4 years.  The system was a 1GHz box with 256MB of RAM.  MySQL puts each database into a seperate directory in a single "data" directory.   Once the old system reached about 10K databases the connection times increased 10X or more.  I attributed the speed problems to a possible filesystem limitation with a large number of files.  That system ran RH9 and either had ext2 or ext3 with no htree patch. 

Recently I bought a new 2.4GHz server with 1GB RAM and much faster drives.  I installed FC4 and am running a 2.6.14 kernel.  I thought that even if my problems were not completely solved, I would at least not see connection times increase until I had many more directory entries (wild guess - 20K).

Since I am writing, you can guess that is not the case.  At 13K directory entries, I am still seeing significantly slower connection times even with the faster hardware and newer software.  

I'm limited in what I can move because MySQL expects everyting in the "data" directory.

My questions are:  1)  I don't know much about htree, but I recall that it is supposed to help in this situation.  How do I tell if it is in use on my system. Is it a kernel module or is it compiled into ext3?   2)  Are there other options for tuneing ext3 performance?

Thanks everyone,

Gary Huntress




More information about the Ext3-users mailing list