The lost art of benchmarking was Re: Reiser4

[Jean Francois Martinez]

On Tue, 2004-09-21 at 04:16, Colin Walters wrote:
> On Mon, 2004-09-20 at 23:12 +0200, Jean Francois Martinez wrote:
> 
> > The other day I had a discussion with the sysadmin of
> > another department and our common position was:  1)  The state of
> > the art in kernel building (ie use of modules and cpu detection by
> > the installer) allows vendors to ship kernels whose
> > performance is close enough to a custom built one the user should
> > not need to bother with recompiling.  2) If the kernel you get out
> > of the box is an underperformer then it is a _bug_.  
> 
> Er, a large part of the LWN audience is kernel and other low-level
> systems developers.  This may seem surprising, but kernel developers
> tend to compile their own kernels, and then install them.
> 

If kernel developers were a _large_ part of LWN's audience then
LWN would have died long ago due to lack of revenue.   Oh and
a still larger portion uses a web browser...


I persist that a proper benchmark should between other things
1) be strongly coupled with what you are trying to measure
2) Try to cover a reasonably large part of the different
aspects of performance.  It is to the user, not to the author
to select the aspects who are important to him and if for some
reason you make a limited test then you should include some
caveats like "kernel compiling is mostly about reading small
files in small or average directories, it doesn't exert this,
this and this"

And now if you want to discuss it further I think we should
do it in private
> 
> ______________________________________________________________________
> -- 
> fedora-test-list mailing list
> fedora-test-list at redhat.com
> To unsubscribe: 
> http://www.redhat.com/mailman/listinfo/fedora-test-list
-- 
Jean Francois Martinez <jfm512 at free.fr>