problems with large directories?

[Eric Sandeen]

Mitch Trachtenberg wrote:
> Hi,
> 
> I have an application that deals with 100,000 to 1,000,000 image files.
> 
> I initially structured it to use multiple directories, so that file
> 123456 would be stored in /12/34/123456.  I'm now wondering if that's
> pointless, as it would simplify things to simply store the file in /123456. 
> 
> Can anyone indicate whether I'm gaining anything by using smaller
> directories in ext3/ext4?  Thanks.
> 
> Mitch 
> 

If you have one file per dir, that's a lot of dirs, and the time to search for
new dir inode locations can get rather expensive as the fs fills, in my experience.

You may also want to toy with setting the "topdir" flag on a dir; new directories
-under- that topdir get spread around the block groups.  New dirs under a non-topdir
tend to stay closer to the parent.

Finally, remember that ext2/3 has a limit of 32000 or so files per dir.
ext4 lifts this restriction.

-Eric