EXT3 and large directories (fwd)

[Christian]

(please reply on-list, so everybody can comment/help)

Matt,

thanks for the details, but apart from mount-option tuning and dir_index 
(which you've already enabled), I dunno why ls(1) would take *hours* to 
stat ~1M files...

out of curiosity: are you able to try a newer kernel? does it change 
anything?

Christian.

---------- Forwarded message ----------

The dir_index was enabled during the filesystem creation.

The directories can have from 0 - 1,000,000 files at any given time

The slowness is on an open of the directory when it is empty, The size
of the directory refers to the directory file itself not the size of the
directories contents. There are no open files in the directory when it
is being stated. I will add that when we do have 50,000 files or more in
the directories a listing is also very slow, can take hours.

Slowness can be 5-10 minutes on a directory open

open("/ems/bigdisk/132", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) =
3
fstat64(3, {st_mode=S_IFDIR|0755, st_size=79298560, ...}) = 0
fcntl64(3, F_SETFD, FD_CLOEXEC)         = 0
getdents64(3, /* 3 entries */, 4096)    = 72
getdents64(3, /* 0 entries */, 4096)    = 0
close(3)                                = 0


Kernel is 2.6.9-34.0.2.ELsmp

I have tried noatime which does speed up reads when there are lots of
files but does not fix the directory issue

These directories are empty except for two file:
drwxr-xr-x  3 vbox132 root  76M Oct  2 10:04 132
drwxr-xr-x  3 vbox151 root 226M Oct  4 17:00 151
drwxr-xr-x  3 vbox229 root  33M Oct  2 10:16 229
drwxr-xr-x  3 vbox235 root 7.5M Oct  2 10:14 235
drwxr-xr-x  3 vbox246 root  52M Sep 30 20:59 246
drwxr-xr-x  3 vbox249 root 1.1M Oct  2 10:04 249

-- 
BOFH excuse #83:

Support staff hung over, send aspirin and come back LATER.

_______________________________________________
Ext3-users mailing list
Ext3-users at redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users