How to do QA

Mon Feb 9 05:38:53 UTC 2004

On Fri, Feb 06, 2004 at 07:01:53AM +0100, Michael Schwendt wrote:
> checks), meeting the requirements of a build system (=> completing the
> build dependencies). However, Fedora Legacy aims at modifying the source
> rpms as little as necessary, so reviewers don't generally need to spend
> any time reading the spec file beyond the diff against the previous
> release.

It has been stated before that Red Hat's build system is different
than fedora.us and fedoralegacy, and not as strict w.r.t. 
BuildRequires.  As a result, many Red Hat packages that we me provide
updates to do not have a proper set of BuildRequires: and hence may
fail to build on our buildsystem.  Therefore, the BuildRequires:
checks are something that should really stay on the QA checklist.

Additionally the ldd checks are an extremely good idea to make sure a
missing BuildRequire isn't causing the resulting binary to miss some
features silently.

I believe it would be good to add some canned set of commands to help
QA testers verify these types of things.  For example, these can be
used on the old and new source and binary RPM's to see what changed:

rpm -qpl redhat-package.rpm > redhat-package
rpm -qpl redhat-package-update.rpm > redhat-package-update
diff -u redhat-package redhat-package-update

Perhaps a similar, but slightly more complicated script could be
devised to do ldd checks.

> checklist don't apply at all to Fedora Legacy. I feel that starting with
> this list as a basis is approaching the "QA problem" from the wrong
> direction. Start at the top and refine the procedures and policies.

I disagree that starting with this list is a bad idea.  Sure, many of
the things can be removed, but many of them are also trivial.  These
policies have been refined over time and it would be a shame to ignore
them and start from scratch.