problems with large directories?
Ric Wheeler
rwheeler at redhat.com
Tue Mar 9 03:14:03 UTC 2010
On 03/08/2010 08:23 PM, Mitch Trachtenberg wrote:
> Hi,
>
> I have an application that deals with 100,000 to 1,000,000 image files.
>
> I initially structured it to use multiple directories, so that file
> 123456 would be stored in /12/34/123456. I'm now wondering if that's
> pointless, as it would simplify things to simply store the file in /123456.
>
> Can anyone indicate whether I'm gaining anything by using smaller
> directories in ext3/ext4? Thanks.
>
> Mitch
>
I think that breaking up your files into subdirectories makes it easier to
navigate the tree and find files from a human point of view. Even better if the
bytes reflect something like year/month/day/hour/min (assuming your pathname has
a date based guid or similar encoding).
You can have a million files in one large directory, but be careful to iterate
and copy them in a sorted order (sorted by inode) to avoid nasty performance
issues that are side effects of the way we hash file names in ext3/4.
Good luck!
Ric
More information about the Ext3-users
mailing list