with dir_index ls is slower than without?

[Ross Boylan]

On Mon, 2008-03-31 at 13:18 +0200, Sebastian Reitenbach wrote:
> Hi Nicolas,
> 
> Nicolas KOWALSKI <niko at petole.dyndns.org> wrote: 
> > "Sebastian Reitenbach" <sebastia at l00-bugdead-prods.de> writes:
> > 
> > > installhost2:~ # time ls -la /mnt/index/ | wc -l
> > >  500005
> > >  
> > >  real 2m41.015s
> > >  user 0m4.568s
> > >  sys 0m6.520s
> > >  
> > >  
> > >  installhost2:~ # time ls -la /mnt/noindex/ | wc -l
> > >  500005
> > >  
> > >  real 0m10.792s
> > >  user 0m3.172s
> > >  sys 0m6.000s
> > >
> > > I expected the dir_index should speedup this a little bit?
> > > I assume I'm still missing sth?
> > 
> > I think the point of dir_index is "only" to quickly find in a large
> > directory a file when you _already_ have its name.
> > 
> > The performance of listing is not its purpose, and as you noted it,
> > even makes performance worse.
> 
> ah, that would explain what I've seen here. 
> 
> after reading your answer, I found this older mail in the archives:
> http://osdir.com/ml/file-systems.ext3.user/2004-09/msg00029.html
See also the
https://www.redhat.com/archives/ext3-users/2007-October/msg00011.html
thread I started about slow directory traversal.  That includes
reference to a library one can load to speed things up sometimes; I was
never clear on exactly how to build and use it (I would need to get a
daemon to use the library) and my only test failed.  I later learned
that tar, my test program, doesn't use the right system calls to
benefit.
> 
> So everything seems to depend on how the application is using the
> filesystem. 
> Picking a single given file might be faster than with a plain ext3, but 
> scanning and opening all files in a directory might become slower. I wanted 
> to use the dir_index for some partitions, like for cyrus imap server, and 
Careful: it was problems backing up a cyrus imap spool that prompted my
question.  I just ran a cyrus backup and it took 35 hours.  Incremental
backups take 3.
> for some other applications. I think I have to benchmark the applications, 
> to see whether they get a speed gain of the dir_index or not.
> 
> kind regards
> Sebastian
> > 
> 
> _______________________________________________
> Ext3-users mailing list
> Ext3-users at redhat.com
> https://www.redhat.com/mailman/listinfo/ext3-users
-- 
Ross Boylan                                      wk:  (415) 514-8146
185 Berry St #5700                               ross at biostat.ucsf.edu
Dept of Epidemiology and Biostatistics           fax: (415) 514-8150
University of California, San Francisco
San Francisco, CA 94107-1739                     hm:  (415) 550-1062