[Fedora-packaging] %check script issues

[Ville Skyttä]

On Tue, 2005-06-28 at 22:52 +0300, Marius Feraru wrote:

> On Jun 27, at 20:45 (+0300), 'Ville Skytt' wrote:
> > Can you come up with a generic case where running the test suite before
> > doing the staged install would be useful?  I cannot at the moment.
> 
> Hm, my mileage varies a lot on this, as I would rephrase it quite the 
> opposite: what would be the generic cases where a staged install is 
> necessary for running its test suite? I cannot at the moment :))

It's not necessary nor does it really have anything directly to do with
it.  But in the vast majority of cases, the test suite must not affect
what gets installed.  I think running tests after the staged install is
a cheap way to be almost certain that it doesn't happen, even if
whatever does the staged install would be buggy as in "too lax
wildcards", or if the test suite would be buggy as in "modifies/removes
files to be installed" or the like.  (Of course this assumes we're
dealing with non-malicious software.  If not, none of this matters
anyway.)


> But anyway, I'll answer your question: perl modules (and obviously not 
> only). Remember the adviced way of install a module:
>    perl Makefile.PL
>    make
>    make test
>    make install
>    ^ as you see, "install" _after_ "test"

Sure, but that's different.  In the above case, you're actually
_installing_ stuff into their final runtime locations, not doing a
staged install the results of which will be installed to their runtime
environment later.  The above advice applies well to the situation it
documents ("how to install these modules"); not doing it that way would
be shooting before asking.  But it applies less well in packaging IMO.


> Now, I'll explain (again) why this testing after install bothers me 
> alot: because I cannot easily repeat testing, previously done with:
>     rpmbuild -bc --short-circuit pkg.spec
> Using "make test" in "%check" nowadays just makes my delay (much) longer,
> as you surely know what a PITA currently are the various scripts run in 
> _spec_install_* (e.g. the dependencies checking).

With %check blocks in use, you can work around that PITA if you want to
stick with rpmbuild alone:

  rpmbuild -bi --short-circuit --define 'install exit 0' foo.spec

Not quite as short as your example above, and may spew errors about
unpackaged files, though.  But doable.  Another way would be to do a
rpmbuild -bc, cd into the build root, and run the test suite manually.


> I'm not 
> against the existence of a "check" script, but against the "policy" to 
> use that block for "make test" & co.

What else would it be useful for?


> > If a test suite of a package does or requires something that is not
> > appropriate (taking into account all situations where it might be run
> > in, generic/minimal build roots and personal workstations or desktops
> > etc.), report a bug against the package.
> 
> Hm, as long as the packager provides clarifying BuildRequires, I see no 
> problem with that.

There are things that a packager may want to test locally but which
cannot be specified as build dependencies, nor even completely written
in the specfile by other means so that they would apply everywhere.

For example, testing database drivers which require a running database
and the configuration to access it, or test suites that require internet
access, or stuff that can be only run as root, or need some files
outside of the build root somewhere, or import GPG keys, to name a few
I've seen.


> > Mileages vary, but I'd suggest at least running the test suite after
> > %install, no matter what way you choose.
> 
> OK, I understand now that I failed explaining my premises in my previous 
> message. I hope now you understood that I'm NOT trying to avoid testing, 
> but just trying to ease packagers' jobs.

Yep, got it.  And I'm just another package monkey explaining one's
understanding of the %check script/block, and opinions on running test
suites as part of rpmbuild in general, FWIW.  I'm slightly advocating
%check over other "homebrew" solutions because the former is a standard
part of rpmbuild which behaves in a certain predictable way (as such),
which cannot be said of the latter without massive coordination efforts
and agreements between packagers.  Computers vs. packagers (humans,
yuk :)) and their opinions, kind of.