[Avocado-devel] Should NRunner support the AVOCADO_* environment variables for non-instrumented tests like the legacy runner?

[Willian Rampazzo]

Double posting for broader coverage.

Hello, Avocado community,

While I was investigating how to implement
https://github.com/avocado-framework/avocado/issues/4742, I reached
some walls that depend on some discussion.

In the legacy runner, the following environment variables are set for
any kind of test:

- AVOCADO_VERSION;
- AVOCADO_TEST_BASEDIR;
- AVOCADO_TEST_WORKDIR;
- AVOCADO_TEST_LOGDIR;
- AVOCADO_TEST_LOGFILE;
- AVOCADO_TEST_OUTPUTDIR.

For **instrumented tests**, those environment variables are set inside
the Test class, independent of runner/nrunner:

https://github.com/avocado-framework/avocado/blob/8c87bfe5e8a1895d77226064433453f158a3ce56/avocado/core/test.py#L874-L883

For **non-instrumented tests**, while it is possible to set
AVOCADO_VERSION, AVOCADO_TEST_BASEDIR, AVOCADO_TEST_LOGDIR, and
AVOCADO_TEST_OUTPUTDIR using the values from runnable config,
AVOCADO_TEST_WORKDIR and AVOCADO_TEST_LOGFILE are not defined.

In the current implementation of nrunner, the information about the
test is exchanged using the messages mechanism, so the
AVOCADO_TEST_LOGFILE does not need to be available to the test or even
exist in some cases.

The AVOCADO_TEST_WORKDIR is a temporary folder created for the test.
It can make use of the folder if needed.

Below is a search **restricted to GitHub** of projects using those
environment variables:

* AVOCADO_VERSION

  - https://github.com/dmonakhov/debian-avocado/blob/fce2e8245bf8311d969f92cc7b6d0569970f35b5/update.sh

* AVOCADO_TEST_BASEDIR

  - No occurrences

* AVOCADO_TEST_WORKDIR

  - No valid occurrences (there is one occurrence from an old QEMU
acceptance test code)

* AVOCADO_TEST_LOGDIR

  - No occurrences

* AVOCADO_TEST_LOGFILE

  - No occurrences

* AVOCADO_TEST_OUTPUTDIR

  - https://github.com/dmonakhov/fio-testsuite-generator/blob/4138a42db5d2b42e364cbd5a791b43333fd68870/templates/run_jobs.sh.j2
  - https://github.com/dmonakhov/vzt-ktest-util/blob/c4e0c8e79360caad351f6220a102688f7c0437b8/vzt-kernel-install
  - https://github.com/dmonakhov/jenkins-pipeline-tests/blob/3c37789af2b66090605921450c84e5c4e466b28a/avocado/xfstests-bld.sh
  - https://github.com/avocado-framework/avocado-misc-tests/blob/9f73c300f825666f544c37734eb660d4ac773759/io/net/ethtool/ethtool_tests.sh
  - https://github.com/chenjing236/avocado-cloudtest/blob/7c1c9790fc24017d9f0bce6462a7271a8ea731f0/cloudtest/openstack/compute.py

Considering users may not be using most of these environment
variables, and given the dilemma of supporting some on nrunner, like
AVOCADO_TEST_WORKDIR and AVOCADO_TEST_LOGFILE, the questions I have to
start a discussion are:

- Should we support all of these environment variables for
**non-instrumented tests** on nrunner?
- Should we support only those already available for
**non-instrumented tests** on runnable config?
- Should we deprecate the use of those environment variables for
**non-instrumented tests** on nrunner?

Feel free to reply to this thread or in the GitHub Discussion
(https://github.com/avocado-framework/avocado/discussions/4751). Both
will be taken into consideration.

Thanks,

Willian