Running rpmlint within mock

[David Malcolm]

On Fri, 2006-07-14 at 19:04 -0400, Jesse Keating wrote:
> On Friday 14 July 2006 18:48, Jason L Tibbitts III wrote:
> > The whole point is that mock already has everything necessary and
> > pregenerated.  It has the chroot just sitting there.  It has the
> > security infrastructure and the procedures for doing things within the
> > chroot already done.  It seems to me to be terribly foolish to
> > duplicate that.
> >
> > How about this: does mock provide (or could it provide) a way for me
> > to install a couple of packages and run a command within the chroot?
> > Or is this relatively simple to do safely without using mock?  Forget
> > about rpmlint; maybe I need this for debugging or to figure out why
> > something isn't building.
> 
> The same software could be used.  Mock can do it, it just isn't designed with 
> this task in mind.
> 
> More to the point, there are two distinct tasks here, one is building the 
> package, the other is testing.  They have some pretty different needs, and 
> trying to do both of them in the same harness isn't the best of ideas.  
> rpmlint is just a test, it leads to more tests, like I described.  What about 
> doing functional tests on things like php packages, where we want to spin up 
> a local apache, browse to some pages, and make sure the content looks right.  
> These are the types of things that can be done with dogtail in a real test 
> environment that really really really shouldn't be done at build time.
I like to classify testing into two types: static vs dynamic

When people refer to "static analysis" they're normally thinking of e.g.
a new compiler plugin which applies some semantic rules to the parse
tree and e.g. decides that you're mutex locking is dodgy (or whatever),
which normally involves either a build-time test, or a rebuild.

I like to generalize the term to any kind of test that doesn't require
you to actually run the code (or install it, for that matter).  This can
include the tests that rpmlint already performs, or all kinds of other
goodness (either on the metadata, or perhaps on the files in the package
payload itself).  So this can happen at some arbitrary time after the
package is built - perhaps as an automatic postprocessing step within
the build system (for both Core and Extras I hope).

So, by contrast, tests that require you to actually run the code, I call
"dynamic tests" - be it a functional test of the kind Jesse described,
or a simple "does the app start without coredumping" test; do upgrades
and downgrades work? (exercising the RPM scriptlets) etc

We're working on opening up parts of our dynamic testing framework and
our dynamic tests, as Will Woods has described in other emails.  I'd
love it if we could hook up a static testing framework as well - and I'm
not sure exactly where it would sit (mock, plague, somewhere else).  The
ideas on the wiki page for a pluggable analysis framework look great.

Hope this helps
Dave