[Avocado-devel] RFC: Parameters as Environment Variables

[Amador Pahim]

On Wed, Jan 10, 2018 at 12:35 PM, Ademar Reis <areis at redhat.com> wrote:
> Hi Amador.
>
> On Tue, Jan 09, 2018 at 01:27:14PM +0100, Amador Pahim wrote:
>> Motivation
>> ==========
>>
>> Avocado creates a tree like object to represent its parameters coming
>> from the varianter. But that tree object can be accessed only by
>> INSTRUMENTED tests. When it comes to non-INSTRUMENTED tests, test
>> writers have to rely on the environment variables created out of that
>> tree structure. The translation from tree object to environment
>> variables has some caveats currently and that's the main focus of this
>> RFC.
>>
>> Reference: https://trello.com/c/4cNAKW6I/1111-discuss-a-way-to-control-the-export-of-parameters-as-environment-variables
>>
>> Currently, the tree object is flattened, creating environment
>> variables with the names of the leaves, disregarding the branch path
>> and also overwriting preexisting environment variables with the same
>> name.
>>
>> Example:
>>
>>     $ cat parameters.yaml
>>     branch1:
>>         foo: bar1
>>     branch2:
>>         foo: bar2
>>
>> With the above yaml file, avocado will generate the following tree:
>>
>>     $ avocado variants -m parameters.yaml --tree -c
>>     Multiplex tree representation:
>>      ┗━━ run
>>           ┣━━ branch1
>>           ┃     → foo: bar1
>>           ┗━━ branch2
>>                 → foo: bar2
>>
>>
>> Even though two parameters have the same name, they are in different
>> branches and have different values. Both parameters can be accessed in
>> INSTRUMENTED tests using:
>>
>>     self.params.get('foo', path='/run/branch1/*')
>>
>> and
>>
>>     self.params.get('foo', path='/run/branch2/*')
>>
>> When it comes to non-INSTRUMENTED tests, i.e. SIMPLE tests, the tree
>> is flattened and the corresponding environment variables are created.
>> In the same example above, a bash script which used the "foo"
>> parameter above would look like this:
>>
>>     #!/bin/bash
>>     echo ${foo}
>>
>> But since we don't use the path information, the $foo is the last
>> parameter with that name from the tree. The previous "foo" value will
>> not be available. And, as mentioned before, a previous environment
>> variable with that name will be overwritten, with obvious
>> consequences.
>>
>
> Ouch...
>
> Perhaps you should deprecate this feature ASAP (provide some sort of
> warning to users of it, or document it as deprecated)?


Yes, I will start the deprecation process. But being that the only
support we have for parameters access right now, the feature can only
be completely dropped after we implement this RFC.

>
>>
>> Proposal
>> ========
>>
>> To avoid parameters collision, one option is to use the path
>> information to create the environment variable name. In the example
>> above, instead of one "foo" variable, we would create a
>> "run_branch1_foo" containing "bar1" and a "run_branch2_foo" containing
>> "bar2".
>>
>> To avoid overwriting the preexisting environment variables, one option
>> is create a more unique variable name, including the "AVOCADO_" string
>> at the beginning. The same example above would then become
>> "AVOCADO_run_branch1_foo" and "AVOCADO_run_branch2_foo", with their
>> respective values.
>
> Right, I think you can call "AVOCADO_" a namespace and we already
> use it for some information such as AVOCADO_VERSION,
> AVOCADO_TEST_LOGDIR, etc.
>
>>
>> The proposal above would improve the situation, while keeping the
>> current user experience, enabling more powerful parameters API for
>> non-INSTRUMENTED tests. Besides the proposals above, one extra feature
>> that we have to consider is to create an environment variable with a
>> machine-parsable content with the serialized varianter object. A good
>> format for that would be JSON, a text format that is completely
>> language independent but uses conventions that are familiar to
>> programmers, which is already used by Avocado to serialize the
>> varianter object in the 'jobdata' module.
>>
>> With that in mind, using the same example, the proposal is to create a
>> variable called "AVOCADO_PARAMETERS" containing:
>>
>>     [{"paths": ["/run/*"],
>>      "variant": [["/run/first_branch", [["/run/first_branch",
>>                                                    "foo",
>>                                                    "bar1"]]],
>>                     ["/run/second_branch", [["/run/second_branch",
>>                                                         "foo",
>>                                                         "bar2"]]]],
>>      "variant_id": "first_branch-second_branch-4cd9"}]
>>
>> That structure would be easily parsed/used by tests written in any language.
>>
>> Note: As an additional resource, we could make public a Python API for
>> loading that serialized structure back in a variant object, so Python
>> users would have the same experience as in INSTRUMENTED tests when
>> writing non-INSTRUMENTED Python tests.
>
> I may be missing some of the last developments, but in my mind,
> tests receive parameters, not variants. Is that still the case?

Yes, my example was not the best one. The structure I presented there
is currently internal and its purpose is to support the Job Replay.

>
> In other words, there would be no references to variants in the
> structure visible to tests (not even variant_id). I also don't
> understand why we have '"paths": ["/run/*"]' there, could you please
> explain?

Indeed, tests are not expected to have access to the variant. Instead
of reusing that internal structure I presented, we should then adjust
this item of the proposal to create a parameters-only structure, with
no variant information. Thanks for the remind.

In that case, the "AVOCADO_PARAMETERS" variable would contain:

{key: value, key2: value2}

In our example, using the current varianter implementation, "key"
would have to be "path"+"param", resulting in:

{"/run/branch1/foo": "bar1", "/run/branch2/foo": "bar2"}

>
> Perhaps you can provide a more complete example, and make it
> consistent with the example from the motivation section. Bonus
> points if it's a real-world one.
>
> I also remember past conversations about parameters potentially
> coming from different sources in hierarchical order (say, via
> hierarchical configuration files, command line options, etc) with
> the varianter being one of the providers.  That's one of the reasons
> why I consider it important to provide parameters to tests, not
> variants.  If we keep it abstract like that, nothing has to change
> in this environment variable feature if we make changes to the
> underlying test parametrization mechanism.
>
> Thanks.
>    - Ademar
>
>>
>> Expected Results
>> ================
>>
>> Mitigate the cases where the parameters can not be accessed by
>> non-INSTRUMENTED tests and avoid collision with preexisting
>> environment variables.
>>
>>
>> Looking forward to read your comments.
>> --
>> apahim
>>
>
> --
> Ademar Reis
> Red Hat
>
> ^[:wq!