[Avocado-devel] Avocado and pytest: are they incompatible ?

[Cleber Rosa]

On Wed, Sep 16, 2020 at 04:54:55AM +0000, Barrientos, Carlo wrote:
> Hi Avocado Devs,
>

Hi Carlo,

>      I have a question about Avocado test-framework and pytest:
>              Is it possible to use Avocado CCIT with a test suite written in python using pytest ?
>
>  I know Avocado is compatible with python unittest but I have not found anything in docs that mentions using pytest w/ Avocado; is that because they are incompatible ?
>

There's no compatibility right now, but adding support to pytest tests
is very much a desirable goal for Avocado.

The easist way to do so would be to hook that support in the "nrunner"
architecture.  It would mean writing a resolver to find/list those tests.
The interface is described here:

  https://github.com/avocado-framework/avocado/blob/82.0/avocado/core/plugin_interfaces.py#L219

The "python-unittest" implementation may serve as a guide:

  https://github.com/avocado-framework/avocado/blob/82.0/avocado/plugins/resolvers.py#L92

With a resolver implemented, "avocado list --resolver
/path/to/a/pytest" should list the tests.  The same resolver would be
used when doing a "avocado run --test-runner=nrunner
/path/to/a/pytest", but the runner itself also needs to be written.
One example of an runner can be seen here:

   https://github.com/avocado-framework/avocado/blob/82.0/avocado/core/nrunner_tap.py

Now, even with a resolver and runner implemented, the CIT plugin would
not work with nrunner, because of "no support for variants yet":

  https://github.com/avocado-framework/avocado/blob/82.0/avocado/plugins/runner_nrunner.py#L235

But this should be addressed shortly.

>      I read the paper, Avocado: Open-Source Flexible Constrained Interaction Testing for Practical Application, Feb 2, 2020 by Richter, Ahmed, Bures and Rosa. and I have been trying to understand some of the Avocado 52.0 documentation in hopes of applying some of the Avocado CCIT principles to my current work in testing software at my company.
>

Avocado 52.0 is *ancient*.  Is there a reason you're using that?

>      The test suite I have is almost all written using pytest naming conventions and decorators the tests require extensive complex fixtures to setup and teardown databases and setup conditions for test cases.  The software is NOT a web application and so Selenium
> is not really an optimal framework for my task.   However, I am finding that pytest is not-quite as flexible and automatically driven as I am accustomed to using at my previous employer.

I see.

>      A few years ago, when I was working for Apple, I also worked on testing but their testing framework was much more like Avocado (or Autotest) in that “reference” data for tests was auto-generated from a “known good” software release and maintenance of the tests was much easier especially for cases more complex than unit test;  complex cases which involve functional and system tests in multiple environments.  I am looking for ways to automate testing as much as possible because manual maintenance of tests and fixtures is quickly growing into an unmanageable task for a small team.
> 
>      So again, my question:
> 
>             Is it possible to use Avocado CCIT with a test suite written in python using pytest ?


Right now, *out of the box*, the answer is no.  Having said that, we
tried hard to simplify the addition of new test types to Avocado in
the nrunner architecture.  Let me know if adding pytest support is
something you're interested, and we could work together on it.

Best regards,
- Cleber.

> 
> 
> Carlo Barrientos
> CAD Methodology - Cirrus Logic
> carlo.barrientos at cirrus.com<mailto:carlo.barrientos at cirrus.com>
> 
> 
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20200922/58fd4ac3/attachment.sig>