[Avocado-devel] RFC: Parameters as Environment Variables

[Ademar Reis]

Hi Amador.

On Tue, Jan 09, 2018 at 01:27:14PM +0100, Amador Pahim wrote:
> Motivation
> ==========
> 
> Avocado creates a tree like object to represent its parameters coming
> from the varianter. But that tree object can be accessed only by
> INSTRUMENTED tests. When it comes to non-INSTRUMENTED tests, test
> writers have to rely on the environment variables created out of that
> tree structure. The translation from tree object to environment
> variables has some caveats currently and that's the main focus of this
> RFC.
> 
> Reference: https://trello.com/c/4cNAKW6I/1111-discuss-a-way-to-control-the-export-of-parameters-as-environment-variables
> 
> Currently, the tree object is flattened, creating environment
> variables with the names of the leaves, disregarding the branch path
> and also overwriting preexisting environment variables with the same
> name.
> 
> Example:
> 
>     $ cat parameters.yaml
>     branch1:
>         foo: bar1
>     branch2:
>         foo: bar2
> 
> With the above yaml file, avocado will generate the following tree:
> 
>     $ avocado variants -m parameters.yaml --tree -c
>     Multiplex tree representation:
>      ┗━━ run
>           ┣━━ branch1
>           ┃     → foo: bar1
>           ┗━━ branch2
>                 → foo: bar2
> 
> 
> Even though two parameters have the same name, they are in different
> branches and have different values. Both parameters can be accessed in
> INSTRUMENTED tests using:
> 
>     self.params.get('foo', path='/run/branch1/*')
> 
> and
> 
>     self.params.get('foo', path='/run/branch2/*')
> 
> When it comes to non-INSTRUMENTED tests, i.e. SIMPLE tests, the tree
> is flattened and the corresponding environment variables are created.
> In the same example above, a bash script which used the "foo"
> parameter above would look like this:
> 
>     #!/bin/bash
>     echo ${foo}
> 
> But since we don't use the path information, the $foo is the last
> parameter with that name from the tree. The previous "foo" value will
> not be available. And, as mentioned before, a previous environment
> variable with that name will be overwritten, with obvious
> consequences.
> 

Ouch...

Perhaps you should deprecate this feature ASAP (provide some sort of
warning to users of it, or document it as deprecated)?

> 
> Proposal
> ========
> 
> To avoid parameters collision, one option is to use the path
> information to create the environment variable name. In the example
> above, instead of one "foo" variable, we would create a
> "run_branch1_foo" containing "bar1" and a "run_branch2_foo" containing
> "bar2".
> 
> To avoid overwriting the preexisting environment variables, one option
> is create a more unique variable name, including the "AVOCADO_" string
> at the beginning. The same example above would then become
> "AVOCADO_run_branch1_foo" and "AVOCADO_run_branch2_foo", with their
> respective values.

Right, I think you can call "AVOCADO_" a namespace and we already
use it for some information such as AVOCADO_VERSION,
AVOCADO_TEST_LOGDIR, etc.

> 
> The proposal above would improve the situation, while keeping the
> current user experience, enabling more powerful parameters API for
> non-INSTRUMENTED tests. Besides the proposals above, one extra feature
> that we have to consider is to create an environment variable with a
> machine-parsable content with the serialized varianter object. A good
> format for that would be JSON, a text format that is completely
> language independent but uses conventions that are familiar to
> programmers, which is already used by Avocado to serialize the
> varianter object in the 'jobdata' module.
> 
> With that in mind, using the same example, the proposal is to create a
> variable called "AVOCADO_PARAMETERS" containing:
> 
>     [{"paths": ["/run/*"],
>      "variant": [["/run/first_branch", [["/run/first_branch",
>                                                    "foo",
>                                                    "bar1"]]],
>                     ["/run/second_branch", [["/run/second_branch",
>                                                         "foo",
>                                                         "bar2"]]]],
>      "variant_id": "first_branch-second_branch-4cd9"}]
> 
> That structure would be easily parsed/used by tests written in any language.
> 
> Note: As an additional resource, we could make public a Python API for
> loading that serialized structure back in a variant object, so Python
> users would have the same experience as in INSTRUMENTED tests when
> writing non-INSTRUMENTED Python tests.

I may be missing some of the last developments, but in my mind,
tests receive parameters, not variants. Is that still the case?

In other words, there would be no references to variants in the
structure visible to tests (not even variant_id). I also don't
understand why we have '"paths": ["/run/*"]' there, could you please
explain?

Perhaps you can provide a more complete example, and make it
consistent with the example from the motivation section. Bonus
points if it's a real-world one.

I also remember past conversations about parameters potentially
coming from different sources in hierarchical order (say, via
hierarchical configuration files, command line options, etc) with
the varianter being one of the providers.  That's one of the reasons
why I consider it important to provide parameters to tests, not
variants.  If we keep it abstract like that, nothing has to change
in this environment variable feature if we make changes to the
underlying test parametrization mechanism.

Thanks.
   - Ademar

> 
> Expected Results
> ================
> 
> Mitigate the cases where the parameters can not be accessed by
> non-INSTRUMENTED tests and avoid collision with preexisting
> environment variables.
> 
> 
> Looking forward to read your comments.
> --
> apahim
> 

-- 
Ademar Reis
Red Hat

^[:wq!