Max Files Per Directory

[Mike McMullen]

----- Original Message ----- 
From: "Tony Nelson" <tonynelson at georgeanelson.com>


At 8:53 AM -0400 7/16/05, Matthew Miller wrote:
>On Sat, Jul 16, 2005 at 08:04:22AM -0400, fredex wrote:
>> Now, for performance reasons, it is often not a good idea ot have
>> many thousands of files in a single directory. As the number of files
>> grows large the time it takes to access a file grows larger. I haven't
>> looked into this on any linux file system, but on other unixes I've
>> observed delays reaching up into the whole-second region when many thousands
>> of files are in a single directory.
>
>Shouldn't be a problem on a new install of ext3 on modern Linux. See
><http://lwn.net/Articles/11481/>. 

I suppose the simple test is to try it. Time (microseconds) per file to run
a script that creates files by touching them, to ls a few near the end, and
to remove all the files; see script at end of transcript.

#files    touch    ls       rm
------    -----    -----    -----
  1000      309      016      054
 10000      331      008      051
100000      326      007      357
200000      330      007     1065

I didn't want to try a million files.  Creation and listing seem to be
linear with the number of files, but rm seems quadratic.  I think this
indicates that in my current 2.6.12 FC3 kernel using Ext3 the directory
data structure is still a list and not a tree.

I expect that some sort of directory hierarchy to limit the number of files
per directory would still be a win.

Transcript follows.  Max line width 114 chars.

------ End of Original Message -------

Tony and Mathew thanks very much for your help!!!

This confirms roughly what I thought would be the case. Tony, thanks
especially for going to the trouble of testing it!

Mike