Expectation Management for Test Releases

[Keith Lofstrom]

This weekend, at the party following LinuxFest Northwest, I found myself
across the table from an avionics software project manager at Boeing. 
He told me of some of his problems with Fedora.

He said he had big problems running Fedora Core 2 Test 1;  he was not
mollified when I tried to point out that one can *expect* huge problems
trying to run test releases.   He insisted that it was entirely
unprofessional to release software that wouldn't even install, and that
using the FC2T1 upgrade process on top of a working FC1 system made a
real hash of it.  He had performed alpha and beta testing on Windows
and never had such problems. 

Now, the room was loud, I am hard of hearing, and I may be misrepresenting
what he said.  He may have meant FC2T2.  I tried to argue that the test
series was not intended to meet his expectations, and had other purposes.
I did not manage to make my point.  Perhaps I had a hard time believing
my own point;  code that doesn't even install DOES look pretty silly. 
And it is hard to argue about the nature of good code development with
a person whose career is based on successfully producing some of the
world's most reliable and mission-critical code.

This is probably a problem of managing expectations.  It is quite easy
to navigate from the top of fedora.redhat.com to the FC2Tx download 
area without encountering a single explicit disclaimer or explanation
of what FC2Tx is for, or how dangerously buggy the test code can be. 
Many people, including professional software people heavily oriented
towards testing, may download this code without understanding what
they are in for.  You can read this mail group for months without
seeing a FAQ;  there are apparently none of the warnings that used to
come with rawhide.  Some big warnings, such as a FAQ that appears often
on this list, and on the http://fedora.redhat.com/download/test.html
"download the test release" page, are needed.   We should let potential
testers know the huge difference between a test release and a general
availability release,  before they start downloads.  This would save
this list from a lot of carping from people like the Boeing manager,
and like me.  State the goals.  Be honest about the downside.

That will cause a lot of potential testers to go away, sure.  It might
also prevent disappointments that result in Linux (and Linux programmers)
being rejected by major software-using organizations.   A decade from
now, I don't want to ride on an airplane controlled by Windows CE-RT,
because in 2004 we pissed off the wrong person at Boeing. 

Let's work on those disclaimers, okay?

Keith

-- 
Keith Lofstrom           keithl at ieee.org         Voice (503)-520-1993
KLIC --- Keith Lofstrom Integrated Circuits --- "Your Ideas in Silicon"
Design Contracting in Bipolar and CMOS - Analog, Digital, and Scan ICs