Re: ext3 performance issue with a Berkeley db application

[Greg Louis <glouis dynamicro on ca>]

On 20030203 (Mon) at 1117:44 -0800, Andrew Morton wrote:

> hum.
> 
> So you have an operation which takes 30 seconds and generates 30 megabytes of
> dirty data.  My wild guess would be that the first five seconds' worth of
> data are splattered all over the disk, and that the holes in that write
> pattern are filled in later in the run.
> 
> So if a filesystem happens to decide to write the data after the first five
> seconds, there is little request merging.  But if the filesystem were to wait
> the whole thirty seconds then voila - all the holes are filled in and there's
> a lot of request merging.
> 
> Well.  It's a theory.  If you mount your data=ordered fs with `-o commit=60',
> what happens?

Recalling the times reported yesterday, in writeback it's 4 minutes and
change; data=ordered, roughly 27 minutes; the commit=60 result is about
the same as commit=5, as follows:

# umount /xtrn
# mount -t ext3 -o data=ordered,commit=60 /dev/sdc1 /xtrn

# time bogofilter -v -s -d /xtrn/db <spam_corpus
# 5898549 words, 14600 messages

real    28m9.866s
user    2m30.750s
sys     0m46.700s


-- 
| G r e g  L o u i s          | gpg public key:      |
|   http://www.bgl.nu/~glouis |   finger greg bgl nu |
| Help free our mailboxes. Include                   |
|        http://wecanstopspam.org in your signature. |