tests as documentations.

[Chris Weyl]

Lately I've started realizing that the test suite of a distribution
can make for excellent examples of how to use something.  E.g. lots of
the Catalyst and CGI-* dists include little "test apps" that exercise
the various modules; Moose and Class-MOP both contain extensive test
suites (with Moose going so far as to say "the tests are still the
best documentation" for certain features), etc, etc.  Even for
something as mundane as Module::Use I found constraints on the module
that weren't mentioned in the documentation.

So, I've started packaging the test suites under %doc (in new packages
and as I have reason to update existing packages), taking pains (as
always) to deliver them without dependencies and execute bits.  This
seemed rather logical to me, as I've always included other docs
(examples/*, doc/*, etc) in %doc and insisted on it in package
reviews.  I'm trying to package them up without regards to _my_
thinking they're useful, as what I think isn't useful someone else
usually does (and that other person is typically me 6 months in the
future).

I figured I'd do it for a while, see what feedback I got, then come
here and seek more feedback, and maybe put something in
PackagingDrafts/Perl to address a best practice concerning it.  (I
already have a "tests can make good documentation... consider
packaging them" in there, with no screams to date.)

So.  Here's what I've been hearing:

1.  Questions mainly related to why the change in practice on my part,
e.g. "why? ... oh, ok."
2.  "Well, it's your package, it's in %doc and conforms to guidelines"
3.  Test suites ought to be executable, and have all their deps met.

#1 was the most popular, followed by #2.  One person strongly feels
#3, and doesn't appear to buy the "tests make good docs" argument
under any circumstances.  As near as I can tell, most people don't
seem to really care one way or the other.

As things stand, I'm inclined to go forward with "encourage optionally
packaging t/ in %docs, following guidelines (no exec, no deps; split
to -docs if the end package is too large)", but I see a couple
different ways this could go as well.

What do you all think?

                                            -Chris
-- 
Chris Weyl
Ex astris, scientia