The lost art of benchmarking was Re: Reiser4

[Luciano Miguel Ferreira Rocha]

On Mon, Sep 20, 2004 at 06:16:35PM +0200, Leonard den Ottolander wrote:
> Hello Jonathan,
> 
> On Mon, 2004-09-20 at 16:31, Jonathan Corbet wrote:
> > > He also compiled the kernel as a benchmark.  99% of times this is the
> > > touchstone for lack of proper thinking in benchmarking.  
> > 
> > Because I benchmarked an actual task that my readers are likely to
> > perform?  
> 
> No, because if you benchmark a file system you shouldn't benchmark a CPU
> intensive task like compiling a kernel as file system performance will
> only have a relatively small impact on the compilation time
> (signal/noise). That sounds like valid criticism to me.

He wasn't benchmarking the machines, he was benchmarking the file
systems.

All else being equal, a usual operation as a fs benchmark is a valid
one.

A simple, idiotic test like flipping 0s and 1s in ram would still be a
valid test, though with much less significance. If a simple memtest
resulted if so very different results if under different filesystems,
then something would be very wrong with one of them.

Regards,
Luciano Rocha

PS: A compilation can be very dependent on file system performance. Not
usually as much as the processor, of course, but stil...

-- 
Consciousness: that annoying time between naps.