[Avocado-devel] [RFC] Recursive Test Discovery [V2]

[Jeff Nelson]

On Fri, 2 Jun 2017 09:26:32 +0200
Amador Pahim <apahim at redhat.com> wrote:

> Motivation
> ==========
> 
> Currently we can discover test classes that does not inherit directly
> from `avocado.Test`. To do so, we rely on the inclusion of a docstring
> (`:avocado: enable`) in the mentioned class. Example below.
> 
> File `/usr/share/avocado/tests/test_base_class.py`::
> 
>     from avocado import Test
> 
> 
>     class BaseClass(Test):
> 
>         def test_basic(self):
>             pass
> 
> File `/usr/share/avocado/tests/test_first_child.py`::
> 
>     from test_base_class import BaseClass
> 
> 
>     class FirstChild(BaseClass):
>         """
>         :avocado: enable
>         """
> 
>         def test_first_child(self):
>             pass
> 
> In the example above, if we ask Avocado to list the tests from
> `test_first_child.py`, `FirstChild.test_first_child` will be listed and
> the `BaseClass.test_basic` won't::
> 
>     $ avocado list test_first_child.py
>     INSTRUMENTED test_first_child.py:FirstChild.test_first_child
> 
> The request is that, in such cases, we have a way to include the
> `BaseClass.test_basic` into the results.
> 
> Proposal
> ========
> 
> To include the parent classes into the discovery results, we have three
> main aspects to consider:
> 
> - How to flag that we want that behaviour?
>   The proposal is the creation of a new docstring `recursive`. Example::
> 
>     class FirstChild(BaseClass):
>         """
>         :avocado: recursive
>         """
>         ...

I like everything about this RFC except for the word "recursive,"
because I don't think what is being described is a recursive process.
Recursion is a technique to "divide a problem into smaller subproblems
of the same type." [1] I don't think that's what is happening here.
Instead, I imagine this as discovering the class hierarchy in an
iterative fashion, from the starting class to the base class.

So if I'm not fond of "recursive" what do I think should be used
instead? Here is a list of words that I put together, some better than
others:

* discovery
* discover
* ancestors
* ancestor-discovery
* hierarchy
* include-hierarchy
* inheritance
* inheritance-walk
* ancestry

It's hard coming up with a descriptive word that describes the process
of discovering the classes in the inheritance hierarchy. The word
"ancestry" is the term I favor the most.

This is not a NACK; maybe it will spark some ideas. If not, that's
OK too.

-Jeff


[1] https://en.wikipedia.org/wiki/Recursion#In_computer_science


> 
> - How deep is the recursion?
>   The proposal is that the recursion goes all the way up to the class
>   inheriting from `avocado.Test`.
> 
> - Will the recursion respect the parents docstrings?
>   The proposal is that the docstrings in the parents are ignored when
>   recursively discovering. Example:
> 
>   File `/usr/share/avocado/tests/test_base_class.py`::
> 
>     from avocado import Test
> 
> 
>     class BaseClass(Test):
>         """
>         :avocado: disable
>         """
> 
>         def test_basic(self):
>             pass
> 
>   File `/usr/share/avocado/tests/test_first_child.py`::
> 
>     from test_base_class import BaseClass
> 
> 
>     class FirstChild(BaseClass):
>         """
>         :avocado: recursive
>         """
> 
>         def test_first_child(self):
>             pass
> 
>   Will result in::
> 
>     $ avocado list test_first_child.py
>     INSTRUMENTED test_first_child.py:FirstChild.test_first_child
>     INSTRUMENTED test_first_child.py:BaseClass.test_basic
> 
> 
> Expected Results
> ================
> 
> The expected result is to provide users more flexibility when creating
> the Avocado tests, being ablr to create a chain of test classes and
> providing only the module containing the last one as a test reference.
> 
> Additional Information
> ======================
> 
> Avocado uses only static analysis to examine the files and this
> feature should stick to this principle in its implementation.
>