[Avocado-devel] Appropriate test+library layout?

[Amador Pahim]

On Tue, Dec 12, 2017 at 5:35 PM, Amador Pahim <apahim at redhat.com> wrote:
> On Sun, Dec 10, 2017 at 10:32 PM, Paul Graydon <paul.graydon at oracle.com> wrote:
>> Hi all,
>>
>> We're looking at using Avocado to carry out functional testing.  Of
>> particular interest is the possibility of providing dimensions for
>> testing (need to repeat the same tests but with one or two slight
>> differences each time).
>>
>> I'm running in to some quirks that indicate I'm either approaching
>> things wrong, or I need to work around it.  I guess I'm seeking advice
>> on good test layout or the best approach to take with avocado.
>>
>> Here's a toy example that reflects the approach we started with:
>>
>> $ find .
>> .
>> ./foo
>> ./foo/__init__.py
>> ./foo/util.py
>> ./tests
>> ./tests/__init__.py
>> ./tests/test_foo.py
>>
>>
>> $ cat foo/util.py
>> def return_true():
>>     print "Returning true"
>>     return True
>>
>>
>> $ cat tests/test_foo.py
>> from foo.util import return_true
>> from avocado import Test
>>
>>
>> class TestFooUtil(Test):
>>
>>     def test_return_true(self):
>>         assert return_true()
>>
>>
>> The __init__.py files are blank.
>>
>> If I run this from ./ it fails because it can't import foo :
>>
>> $ avocado run .
>>
>> ...
>>
>> ImportError: No module named foo.util
>>
>> ...
>>
>>
>> If I drop sys and print sys.path in to test_foo.py it looks like avocado
>> is executing the code from within the tests directory, so imports would
>> need to be relative.  I can meddle with the path to make things work, if
>> needs be.
>>
>> If I quickly modify that test_foo.py file and turn it in to a standard
>> unittest, I can invoke it from the parent directory and have it import
>> correctly:
>>
>> $ python -m unittest -c tests.test_foo
>> Returning true
>> .
>> ----------------------------------------------------------------------
>> Ran 1 test in 0.000s
>>
>> OK
>>
>> Or allow it to discover the tests itself:
>>
>> $ python -m unittest discover
>> Returning true
>> .
>> ----------------------------------------------------------------------
>> Ran 1 test in 0.000s
>>
>> OK
>>
>>
>> Is there a way to approach this using avocado that results in it finding
>> all the tests when run from a directory, but executing the python from
>> that context, instead of the directory where it found the tests?
>> Should I be laying things out in a different fashion?  I don't see
>> anything applicable in the documentation, but there's every good chance
>> I'm just reading it wrong!
>>
>> There's lots of tests planned that we're hoping to keep structured
>> cleanly under various subdirectories, and a fair bit of utility code
>> that is going to be necessary to give us the test / dimensionality we
>> really need.
>>
>> Any advice would be appreciated.
>
> Well, there are many caveats on doing relative imports. We can dig
> into the details on how Avocado executes the tests and so on, but to
> be objective, my recomendation is that you install your lib and use it
> as an absolute import. Example:
>
> $ find .
> .
> ./foo
> ./foo/__init__.py
> ./foo/util.py
> ./tests
> ./tests/test_foo.py
> ./setup.py
>
> $ cat setup.py
> #!/usr/bin/env python
>
> from setuptools import setup, find_packages
>
>
> setup(name='avocado-foo',
>       description='Avocado Foo Utils',
>       version='1.0',
>       packages=['foo'],
>       include_package_data=True,
>       )
>
> $ cat tests/test_foo.py
> from foo.util import return_true
> from avocado import Test
>
>
> class TestFooUtil(Test):
>
>     def test_return_true(self):
>         assert return_true()
>
>
> $ python setup.py install --user
> running install
> running bdist_egg
> running egg_info
> <..snip..>
> Installed /home/apahim/.local/lib/python2.7/site-packages/avocado_foo-1.0-py2.7.egg
> Processing dependencies for avocado-foo==1.0
> Finished processing dependencies for avocado-foo==1.0
>
> And run the tests from anywhere:

$ avocado run tests/test_foo.py
JOB ID     : 2f14b60bc82ef54d046b3bb4c7556a4a523488f7
JOB LOG    : /home/apahim/avocado/job-results/job-2017-12-12T17.35-2f14b60/job.log
 (1/1) tests/test_foo.py:TestFooUtil.test_return_true: PASS (0.02 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
| CANCEL 0
JOB TIME   : 1.04 s
JOB HTML   : /home/apahim/avocado/job-results/job-2017-12-12T17.35-2f14b60/results.html

$ cd tests/
tests$ avocado run test_foo.py
JOB ID     : 4cc1f5660c7923226310253d89b6c5d9a14c4a2b
JOB LOG    : /home/apahim/avocado/job-results/job-2017-12-12T17.35-4cc1f56/job.log
 (1/1) test_foo.py:TestFooUtil.test_return_true: PASS (0.04 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
| CANCEL 0
JOB TIME   : 0.89 s
JOB HTML   : /home/apahim/avocado/job-results/job-2017-12-12T17.35-4cc1f56/results.html


>
>
>
>>
>> Paul
>>
>>