[Avocado-devel] RFC: Parameters as Environment Variables

[Amador Pahim]

On Wed, Jan 10, 2018 at 6:39 PM, Lukáš Doktor <ldoktor at redhat.com> wrote:
> Dne 9.1.2018 v 13:27 Amador Pahim napsal(a):
>> Motivation
>> ==========
>>
>> Avocado creates a tree like object to represent its parameters coming
>> from the varianter. But that tree object can be accessed only by
>> INSTRUMENTED tests. When it comes to non-INSTRUMENTED tests, test
>> writers have to rely on the environment variables created out of that
>> tree structure. The translation from tree object to environment
>> variables has some caveats currently and that's the main focus of this
>> RFC.
>>
>> Reference: https://trello.com/c/4cNAKW6I/1111-discuss-a-way-to-control-the-export-of-parameters-as-environment-variables
>>
>> Currently, the tree object is flattened, creating environment
>> variables with the names of the leaves, disregarding the branch path
>> and also overwriting preexisting environment variables with the same
>> name.
>>
>> Example:
>>
>>     $ cat parameters.yaml
>>     branch1:
>>         foo: bar1
>>     branch2:
>>         foo: bar2
>>
>> With the above yaml file, avocado will generate the following tree:
>>
>>     $ avocado variants -m parameters.yaml --tree -c
>>     Multiplex tree representation:
>>      ┗━━ run
>>           ┣━━ branch1
>>           ┃     → foo: bar1
>>           ┗━━ branch2
>>                 → foo: bar2
>>
>>
>> Even though two parameters have the same name, they are in different
>> branches and have different values. Both parameters can be accessed in
>> INSTRUMENTED tests using:
>>
>>     self.params.get('foo', path='/run/branch1/*')
>>
>> and
>>
>>     self.params.get('foo', path='/run/branch2/*')
>>
>> When it comes to non-INSTRUMENTED tests, i.e. SIMPLE tests, the tree
>> is flattened and the corresponding environment variables are created.
>> In the same example above, a bash script which used the "foo"
>> parameter above would look like this:
>>
>>     #!/bin/bash
>>     echo ${foo}
>>
>> But since we don't use the path information, the $foo is the last
>> parameter with that name from the tree. The previous "foo" value will
>> not be available. And, as mentioned before, a previous environment
>> variable with that name will be overwritten, with obvious
>> consequences.
>>
>>
>> Proposal
>> ========
>>
>> To avoid parameters collision, one option is to use the path
>> information to create the environment variable name. In the example
>> above, instead of one "foo" variable, we would create a
>> "run_branch1_foo" containing "bar1" and a "run_branch2_foo" containing
>> "bar2".
>>
>
> This can become really long (is there a limit of variable name length in bash?)
>
>> To avoid overwriting the preexisting environment variables, one option
>> is create a more unique variable name, including the "AVOCADO_" string
>> at the beginning. The same example above would then become
>> "AVOCADO_run_branch1_foo" and "AVOCADO_run_branch2_foo", with their
>> respective values.
>>
>
> I don't like this because people use this to control programs/frameworks that rely on pre-defined env variables. For example people might want to set `DEBUG=5` or `LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1` on some variants via parameters.

This is not an Avocado feature. If people are relying on our
parameters system to create pre-defined variables, they are taking
advantage (or abusing) a side effect of our current implementation. If
we were to support this use case, we should create those env.
variables for INSTRUMENTED tests as well. Right now, we are not.

>
> What I can imagine is to do this by default, but also allow saying some variable should be passed as is, for example:
>
>     FOO=VALUE
>     FOO_PASSTHROUGH=1
>
> which should produce:
>
>     FOO=VALUE
>     AVOCADO_FOO=VALUE
>     AVOCADO_FOO_PASSTHROUGH=1
>
>> The proposal above would improve the situation, while keeping the
>> current user experience, enabling more powerful parameters API for
>> non-INSTRUMENTED tests. Besides the proposals above, one extra feature
>> that we have to consider is to create an environment variable with a
>> machine-parsable content with the serialized varianter object. A good
>> format for that would be JSON, a text format that is completely
>> language independent but uses conventions that are familiar to
>> programmers, which is already used by Avocado to serialize the
>> varianter object in the 'jobdata' module.
>>
>> With that in mind, using the same example, the proposal is to create a
>> variable called "AVOCADO_PARAMETERS" containing:
>>
>>     [{"paths": ["/run/*"],
>>      "variant": [["/run/first_branch", [["/run/first_branch",
>>                                                    "foo",
>>                                                    "bar1"]]],
>>                     ["/run/second_branch", [["/run/second_branch",
>>                                                         "foo",
>>                                                         "bar2"]]]],
>>      "variant_id": "first_branch-second_branch-4cd9"}]
>>
>
> This would be nearly impossible to use directly from BASH and so I see this as integration feature, therefor I'd wait for users to ask for it. And when that time comes we'd have better understanding of their needs (probably I'd prefer storing it in file rather than env variable, but can't really tell why...).
>
> If we decide we needed a way to get full representation, I'd go with variables available in a dir structure. On TMPFS it should be fairly quick to create it and easy to access via `find`. Anyway I still think it should be disabled by default and only available via settings or a special param.

Getting full representation is the main point of this RFC. Using tmpfs
is indeed a good alternative.

>
>> That structure would be easily parsed/used by tests written in any language.
>>
>> Note: As an additional resource, we could make public a Python API for
>> loading that serialized structure back in a variant object, so Python
>> users would have the same experience as in INSTRUMENTED tests when
>> writing non-INSTRUMENTED Python tests.
>>
>> Expected Results
>> ================
>>
>> Mitigate the cases where the parameters can not be accessed by
>> non-INSTRUMENTED tests and avoid collision with preexisting
>> environment variables.
>>
>
> I see only one big issue and that is accidental overriding of special variable names (like PATH="my_path"), but still our system should allow that on request (people might want to override PATH for certain variants).
>
> As for the clashes I believe people who expect env variables use simple variant files. Eventually I'd prefer the dir-like structure to provide the variables as that can be easily searched by `find` and `fgrep` but only if that's an optional feature as it'd be costly.
>
> As for the json, yes, it can be useful to integrate with other languages, but I'd wait for user request.

Ok, let's keep this part of the RFC on hold and focus in having the
current issues addressed in an implementation that can fully represent
the parameters. I'm working on a V2 of this RFC.

>
> Regards,
> Lukáš
>
>
>>
>> Looking forward to read your comments.
>> --
>> apahim
>>
>
>