rfe: simple framework for post-build checks

Bill Nottingham notting at redhat.com
Wed Mar 15 03:24:55 UTC 2006


The build system should have a framework where simple post-build
checks can run. Simple design would be:

- tests are collected in a package of scripts (or in CVS)
- packagers can add extra ones for their packages
- tests have access to:
  - the build logs
  - the name of the user who submitted the build
  - the owner of the package (if not the same)
  - the previous version of the package
  - the repodata for the repo being built for
- tests have a defined return status, such as
  0 - everythings OK. Carry on, sir!
  1 - some information here
  7 - someone should review this and approve before pushing
      the package
  11 - the package is bad, please throw away

These tests could then be:
- e-mailed to the builder
- e-mailed to the owner
- posted on the web

Doing this allows for simple, automated, QA; it can check that
things that are caught in the package review for initial import
are then caught later if they regress, and it can catch other
things.

Examples of tests:
- test for files owned by the compile user
- test for files with executable stack
- look for specific compiler warnings in the build output, such as:
    ... will always overflow destination buffer ...
    ... warning: format argument XX unused before used argument XX ...
    *** buffer overflow detected ***
    ... warning: ignoring return value of 'realloc' ...
- check for introduced multilib conflicts
- check the scriplets for:
  - correct use of chkconfig
  - correct use of useradd or equivalents
  - excess complication (warning: package %post is 150 lines)
- using the repo data:
  - check the package for broken deps
  - check the repository for induced breakage
- using previous version of the same package:
  - check for added/removed files
  - check for added/removed/changed dependencies
  - check for introduction of new setuid files
  - check that symbols changed in libraries without changing soname
- heck, just run the package through rpmlint and mail it out
  (or check it against a known list of exceptions for the package)

I'm sure people could come up with a lot more. By making it
a simple framework, it allows for easier contribution.

Bill




More information about the Fedora-buildsys-list mailing list