[Avocado-devel] RFC: Avocado multiplexer plugin

[Lukáš Doktor]

Hello guys,

the purpose of this RFC is to properly split and define the way test 
parameters are processed. There are several ways to split them apart, 
each with some benefits and drawbacks.


Current params process
======================

`tree.TreeNode` -> Object allowing to store things (environment, 
filters, ...) in tree-structure
`multiplexer.Mux` -> Interface between job and multiplexer. Reports 
number of tests and yields modified test_suite templates
`multiplexer.MuxTree` -> Object representing part of the tree from the 
root to leaves or another multiplex domain. Recursively it creates 
multiplexed variants of the full tree.
`multiplexer.AvocadoParams` -> Params object used to retrieve params 
from given path, allows defining several domains for relative paths 
matching defined as `mux_path`s.
`multiplexer.AvocadoParam` -> Slice of the `AvocadoParams` which handles 
given `mux_path`.
`test.Test.default_params` -> Dictionary which can define test's default 
values, it's intended for removal for some time.


Creating variants
-----------------

1. app
2. parser -> creates the root tree `args.default_avocado_params = 
TreeNode()`
3. plugins.* -> inject key/value into `args.default_avocado_params` (or 
it's children). One example is `plugins.run`'s --mux-inject, the other 
is `avocado_virt`'s default values.
4. job -> creates multiplexer.Mux() object
     a. If "-m" specified, parses and filters the yaml file(s), 
otherwise creates an empty TreeNode() called `mux_tree`
     b. If `args.default_avocado_params` exists, it merges it into the 
`mux_tree` (no filtering of the default params)
     c. Initializes `multiplexer.MuxTree` object using the `mux_tree`
5. job -> asks the Mux() object for number of tests
     a. Mux iterates all MuxTree variants and reports `no_variants * 
no_tests`
6. runner -> iterates through test_suite
     a. runner -> iterates through Mux:
         i.  multiplexer.Mux -> iterates through MuxTree
         	* multiplexer.MuxTree -> yields list of leaves of the `mux_tree`
         ii, yields the modified test template
     b. runs the test template:
         i. Test.__init__: |
             if isinstance(params, dict):
                 # update test's default params
             elif params is None:
                 # New empty multiplexer.AvocadoParams are created
             elif isinstance(params, tuple):
                 # multiplexer.AvocadoParams are created from params
7. exit

AvocadoParams initialization
----------------------------

     def __init__(self, leaves, test_id, tag, mux_path, default_params):
         """
         :param leaves: List of TreeNode leaves defining current variant
         :param test_id: test id
         :param tag: test tag
         :param mux_path: list of entry points
         :param default_params: dict of params used when no matches found
         """

1. Iterates through `mux_path` and creates `AvocadoParam` slices 
containing params from only matching nodes, storing them in 
`self._rel_paths`
2. Creates `AvocadoParam` slice containing the remaining params, storing 
them in `self._abs_path`

Test params
-----------

     def get(self, key, path=None, default=None):
         """
         Retrieve value associated with key from params
         :param key: Key you're looking for
         :param path: namespace ['*']
         :param default: default value when not found
         :raise KeyError: In case of multiple different values (params 
clash)
         """

1. Tries to obtain the (key,  path, default) from cache
2. Tries to get the (key, path) from `self._rel_paths`
3. When the `path` is abs_path it tries to get the param from 
`self._abs_path`
4. Looks into the Test's default params dictionary
5. Returns `default`

Overview
--------

Basically now there are 3 components:

1. params parser (yaml => tree)
2. variants generator (tree => iterator)
3. test parameters (key => value from variant values)

and we need to decide how do we want to split it and what should be part 
of the plugin API and what core API.


Plugin parser->tree
===================

We can say the `tree` is the actual params API and we could only allow 
producing custom parsers to construct the tree. This would be the 
simplest method as it only requires us to move the yaml parsing into the 
module and the `AvocadoParams` would stay as they are. The cons is that 
the plugin writers would only be able to produce params compatible with 
the tree structure (flat, or tree-like).

If we decide to chose this method, we can keep the current avocado 
arguments and only allow replacing the parser plugin, eg. by 
`--multiplex-plugin NAME`. Alternatively we might even detect the 
suitable plugin based on the multiplex file and even allow combining 
them (`.cfg vs. .yaml, ...)

The plugin would have to support:

* parse_file(FILE) => tree_node
* check_file(FILE) => BOOL	// in case we want automatic detection of 
file->plugin

Plugin parser->variant
======================

This would require deeper changes, but allow greater flexibility. We'd 
also have to chose whether we want to allow combinations, or whether the 
plugin should handle the whole workflow. I don't think we should allow 
combinations as that would imply another convention for storing the 
parsed results.

The user would configure in config or on cmdline which plugin he wants 
to use and the arguments would stay the same (optionally extended by the 
plugin's arguments)

The plugin would have to support:

* configure(parser)	// optional, add extended options like --mux-version
* parse_file(FILE)	// does not return as it's up to plugin to store the 
results
* inject_value(key, value, path)	// used by avocado-virt to inject 
default values
* __len__()			// Return number of variants (we might want to extend this 
to accept TEMPLATE and allow different number of variants per TEMPLATE. 
That is currently not supported, but it might be handy
* itervariants(TEMPLATE)	// yields modified TEMPLATE with params set in 
AvocadoParams understandable format


Plugin AvocadoParams
====================

I don't think we should make the AvocadoParams replaceable, but if we 
want to we should strictly require `params.get` compatibility so all 
tests can run seamlessly with all params. Anyway if we decided to make 
AvocadoParams replaceable, then we can create a proxy between the params 
and the plugin.


Conclusion
==========

I'm looking forward to cleaner multiplexer API. I don't think people 
would like to invest much time in developing fancy multiplexer plugins 
so I'd go with the `parser->tree` variant, which allows easy 
extensibility with some level of flexibility. The flexibility is for 
example sufficient to implement cartesian_config parser.

As for the automatic detection, I donẗ like the idea as people might 
want to use the same format with different custom tags.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20160630/b2b2d04c/attachment.sig>