ext3 with maildir++ = huge disk latency and high load

[Ted Ts'o]

On Sat, Sep 24, 2011 at 09:46:49PM +0400, Andrey wrote:
> Sure, indexing is on by default on Debian ext3. I think I'll try to
> test some cases an run bonnie++ on freesh HP server with the same
> configuration.

For really gargantuan directories, indexing definitely hurts when you
do a readdir+stat (i.e. /bin/ls -sF) or readdir+unlink (i.e., rm -rf)/

> Also I have maildir with more than 10000 messages an don't have
> timesouts and access problesm via IMAP to it, that's strange.

That's probably because this problem can be worked around by doing a
readdir, then sorting by the inode number (d_ino), and the doing the
stat or unlink.  Some programs, especially those that expressly deal
with Maildir directories, have this optimization already there.

I also have a LD_PRELOAD hack that can be used to demonstrate why
putting this is a good idea.  You can google for spd_readdir and find
it.  I'll also put the latest version of it in the contrib directory
in e2fsprogs for the next release.

    	     		       	    		 	 - Ted