tests as documentations.

[Chris Weyl]

On 6/7/07, Christopher Hicks <chicks at chicks.net> wrote:
> On Wed, Jun 06, 2007 at 01:45:18PM -0700, Chris Weyl wrote:
> > What do you all think?
>
> Some way to run the test suite would cure my last gripe with replacing CPAN.pm with rpms.  Having the files under %doc is fine, but include a script (exec'd or not) that installs dependant rpms for the test, sets the permissions usefully, and then runs the test suite.  Having them there as docs is good and all, but not having some semipainless standard mechanism for making that more useful would be sad.
>
> Another solutions would be to split out a test package with the dependancies elucidated, but my sense is that that sort of thing doesn't follow the spirit people are doing things in rpmland these days.  My sense may be wrong, but if it isn't why is that?

To be honest, I'd kinda like to see something along these lines as
well.  I've been sticking to my mantra of "tests can make good docs"
-- because they do -- but it is awfully nice to be able to "cd %doc;
prove t/*.t" and show the dist working as expected, using the same
test suite the buildsys used.  Or, to be tested at all -- some
packages have display/network deps that can't be satisfied in mock and
are conditionally disabled.  ...or require login ids (a la
WWW::Myspace) for the full suite.

So what would be a good way to do something like this?  Brain dump follows:

1. little README.t also in %doc, saying something like "you need
Test::More, run as prove -I t/lib t/*.t"
2.  little (non-exec scriptie) in %doc to do much the same
3. split tests into a -test subpackage, with deps et al and a little
scriptie derived from {Makefile,Build}.PL
4. a totally separate package that knows where to find tests and
handles kicking them off

#1 would seem to be the easiest.  If we do anything beyond 1 & 2, I
suspect we're going to get some pressure to move them out of %doc
(which is just fine).

#3 I'm not a fan of; it would seem to have the greatest work for the
least return....  plus decentralize the logic.

#4 I'm a fan of.  Another package providing a way to "discover" and
use tests included via other packages...  makes it easy to 1)
optionally include tests as tests in packages (whereever they're
installed), 2) update the framework easily, and 3) provide a
consistent interface.

That all being said, #0 would be: "prove t/*.t" still works nicely in
most cases :)

                                     -Chris
-- 
Chris Weyl
Ex astris, scientia