Ancient Very slow directory traversal thread

[Andreas Dilger]

On Dec 04, 2007  10:52 -0800, Ross Boylan wrote:
> On Mon, 2007-12-03 at 14:37 -0500, Rashkae wrote:
> > I just came across your message to a mailing list here:
> [message concerned it taking hours to go through directories in a mail
> spool on ext3]
> > 
> > https://www.redhat.com/archives/ext3-users/2007-October/msg00019.html
> > 
> > This might be a problem you resolved for yourself a long time ago, but I 
> > thought you might be interested to know that Theodore's spd_readdir 
> > library works great with star (even though I also cannot get it to work 
> > with tar or even du).
>
> That's interesting.  Since I couldn't get it to work with tar, got no
> response, and wasn't sure how or if to get it to work with the daemon to
> that really needed it, I haven't made and progress.
>
> I wonder what determines whether the library helps or hurts.

Maybe it depends if the app is using normal readdir() calls, or is
maybe implementing the directory traversal itself?

> I also just noticed that e2fsck has an option, -D, to optimize
> directories.  The man page says this will reindex directories.  Does
> anyone know if that could help?

That will compress empty space from directories and rebuild the hash
table for directories that are not indexed (e.g. older dirs created
before DIR_INDEX feature was enabled in fs).

It will keep them in hash order so it won't help this issue.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.