An introduction of the new cheerleader...

[Michael Schwendt]

On Thu, 29 Jan 2004 19:57:54 +0800, Ow Mun Heng wrote:

> Newbie Q.. What constitutes Good Testing? Just simply installing it
> and then running it for a couple of days etc... and that's it??

Define "Good". ;) 

Yes, certainly, installing the software (which *may* require the tester to
build a package from src.rpm first) and using it for a period of time is
better already than not trying it at all. It's possible to build a package
which installs, but doesn't work at all [e.g. crashes immediately for
people other than the packager ;) or refuses to start due to missing
files/dependencies]. 

Knowing that the software works for somebody actually is an important
step. People who can report something like that are helpful and could
build teams with people who would perform a lower level technical review
in order to avoid packaging problems. (Uninstallation and an upgrade
attempt can be important, too, btw.)

And of course, there's always the opportunity to publish such a tested
package into the "testing" repository where more people would likely try
it. Ugly packaging details could be fixed later.

With every fresh upstream release--even if it's described as a bug fix
release--there can be new bugs, e.g. fixing-attempts which have
side-effects. Sometimes, waiting a few days before a fresh package is
published adds "community QA", that means the upstream project is
informed by someone about a bug which they have overlooked and which you
(the tester) have not run into [yet] and that results in a quick bug-fix
release.

Ultimately, the testing scheme would be bullet-proof and verify that all
of a program's old and new features work as expected and together with all
possible configurations and all that over an extensive testing period.

That's not feasible, considering how many options some software has. But
one can try out key features and, depending on whether a package update
targets a stable distribution, have a look that an upgrade doesn't
obsolete any features unexpectedly. With new packages one can examine the
packaging (e.g. for completeness, file permissions, feature set), the
pre-configuration and integration work.

--