Ancient Very slow directory traversal thread
adilger at sun.com
Wed Dec 5 23:47:38 UTC 2007
On Dec 04, 2007 10:52 -0800, Ross Boylan wrote:
> On Mon, 2007-12-03 at 14:37 -0500, Rashkae wrote:
> > I just came across your message to a mailing list here:
> [message concerned it taking hours to go through directories in a mail
> spool on ext3]
> > https://www.redhat.com/archives/ext3-users/2007-October/msg00019.html
> > This might be a problem you resolved for yourself a long time ago, but I
> > thought you might be interested to know that Theodore's spd_readdir
> > library works great with star (even though I also cannot get it to work
> > with tar or even du).
> That's interesting. Since I couldn't get it to work with tar, got no
> response, and wasn't sure how or if to get it to work with the daemon to
> that really needed it, I haven't made and progress.
> I wonder what determines whether the library helps or hurts.
Maybe it depends if the app is using normal readdir() calls, or is
maybe implementing the directory traversal itself?
> I also just noticed that e2fsck has an option, -D, to optimize
> directories. The man page says this will reindex directories. Does
> anyone know if that could help?
That will compress empty space from directories and rebuild the hash
table for directories that are not indexed (e.g. older dirs created
before DIR_INDEX feature was enabled in fs).
It will keep them in hash order so it won't help this issue.
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.
More information about the Ext3-users