Large File Copy to Large ext3 RAID5 Array Often Stalls

[Calin Brabandt]

Thanks Alex!  I'm running 2.6 from a new install of Fedora Core 2 so it's my
understanding that the htree fixes were there all along--unless they came
with one of the Core 2 kernel updates.  Samba may be responsible for some of
my problems, but because I had the same problems with ftp and I couldn't
correlate the trouble to anything on the client machines or network, I was
ready to blame ext3 or the RAID5.  Things seem better now that I recreated
the directory structure from the server console and moved the files to the
new subdirectories, rather than copying the subdirectories from the Samba
clients.

Cal

-----Original Message-----
From: Alex Bligh [mailto:alex at alex.org.uk] 
Sent: Sunday, August 01, 2004 11:35 AM
To: Calin Brabandt; ext3-users at redhat.com
Cc: Alex Bligh
Subject: RE: Large File Copy to Large ext3 RAID5 Array Often Stalls



--On 31 July 2004 14:36 -0700 Calin Brabandt <calinb at comcast.net> wrote:

> Perhaps this IS a Samba problem.  I created a new directory structure on
> my RAID5 array from the Linux console and move its contents of large
> files to the new subdirectory.  Although I need to do more testing, the
> problem seem to have vanished--at least when copying new files from my
> network Samba clients to the new subdirectory.

I am *FAR* from an expert, but I seem to remember that samba isn't
particularly efficient at scanning directories, not least because of
case-insensitive filename matching.

You didn't say what kernel you were using, but if it is 2.4, then ext3 is
also not efficient at holding directories with large numbers of files
(which is what I think you said you had). It may be that the two problems
compound to give you very slow performance.

You can try the htree patch for your kernel, then ensure the directory gets
indexed - IIRC mv the directory out the way, create a new directory when
htree is turned on (which will then be created with the relevant htree
index), and mv each of the files in the old directory back into the new
directory.

If you were using 2.4, but in the intervening period you've upgraded to 2.6
(which has the htree fixes in), I note you've just done effectively that,
which may be why the problem has disappeared - just a guess.

Alex