EXT3 and large directories

[Alex Bligh]

--On 05 October 2006 13:02 -0400 Theodore Tso <tytso at mit.edu> wrote:
>> >The issue is over time the directory scans get extremely slow
>> > even if the directories are empty. I have noticed the directories can
>> > range in size from 4k - 100M even when they are empty.
...
>> No way to fix this w/o offline e2fsck -fD.  ext3 doesn't shrink
>> directories when deleting files.
...
> Historically this is a pretty common restriction in Unix filesystems.
> If someone cared enough, it would be possible to change ext3/4 to
> release directory blocks when they are empty, but no one has found it
> important enough to create such a patch.

I had sort of assumed this wouldn't be a problem after htree was
incorporated as far speed, as opposed to size is concerned - and speed was
the original poster's problem, not size on disk. Does that imply there is
still some linear searching going on, or that htree is not "enough" to
speed up the searches.

How do the deleted entries get reused? EG if I have a mail spool
application, where a given directory has around 100,000 files in at any
time, and they are periodically deleted by age in batches of (say) 10,000
such that the number in the directory never exceeds 100,000, does the size
of the directory just keep growing for ever? Or do newly created directory
entries take up the space in the directory of old ones (assume all the
filenames are unique).

Alex