Large File Copy to Large ext3 RAID5 Array Often Stalls

[Alex Bligh]

--On 31 July 2004 14:36 -0700 Calin Brabandt <calinb at comcast.net> wrote:

> Perhaps this IS a Samba problem.  I created a new directory structure on
> my RAID5 array from the Linux console and move its contents of large
> files to the new subdirectory.  Although I need to do more testing, the
> problem seem to have vanished--at least when copying new files from my
> network Samba clients to the new subdirectory.

I am *FAR* from an expert, but I seem to remember that samba isn't
particularly efficient at scanning directories, not least because of
case-insensitive filename matching.

You didn't say what kernel you were using, but if it is 2.4, then ext3 is
also not efficient at holding directories with large numbers of files
(which is what I think you said you had). It may be that the two problems
compound to give you very slow performance.

You can try the htree patch for your kernel, then ensure the directory gets
indexed - IIRC mv the directory out the way, create a new directory when
htree is turned on (which will then be created with the relevant htree
index), and mv each of the files in the old directory back into the new
directory.

If you were using 2.4, but in the intervening period you've upgraded to 2.6
(which has the htree fixes in), I note you've just done effectively that,
which may be why the problem has disappeared - just a guess.

Alex