Cooperative Bug Isolation for Fedora Core 5

[Ben Liblit]

Will Woods wrote:
> Well said!  I'd say CBI does interactive testing (with nicely automated
> results reporting), while the automated test suite is non-interactive.

Agreed.  It's also worth pointing out that a hand-crafted test suite 
typically has both known inputs and known correct outputs.  CBI doesn't 
know what the user's inputs will be, and therefore uses a simpler form 
of known correct output: namely, that crashing is incorrect no matter 
*what* the input was.

Of course, crashing is just the simplest common baseline that everyone 
can agree is bad.  If you have a more specialized way of recognizing bad 
program behavior we can use that too: assertion failures, g_error() 
calls, malformed output, etc.  All we really need is a way to label a 
given run as "good" or "bad".

> This is brilliant.  I'd love to get you guys involved more directly in
> Fedora testing.  What can we do to help?

Thanks!  We'd welcome the opportunity to partner up with Red Hat / 
Fedora in some way.  I think what would be needed is someone inside the 
Red Hat / Fedora organization who wants to champion this.  Someone who 
is enthusiastic about the idea and able to convince others to buy in. 
The CBI team is small, but we can provide expertise and help you work 
though any problems that may arise, since we feel highly motivated to 
see this approach get adopted more broadly.  What we'd really need is an 
inside champion to work with on this project.  Will, are you that person?