[Avocado-devel] RFC: Plugin execution order

[Jeff Nelson]

On Thu, Sep 29, 2016 at 01:24:22PM -0300, Cleber Rosa wrote:
>As detailed in the following card:
>
>  https://trello.com/c/oWzrV48E/837-execution-order-support-in-plugins
>
>It should be possible to specify a custom order for plugins to be
>executed by setting specific configuration.
>
>The first observed approach would be to create a section
>called `[plugins.<type>]` where the `<type>`  conforms to the
>description on fully qualified plugin names given here:
>
>
>https://github.com/avocado-framework/avocado/pull/1495/commits/193a10ce98cb5747395eefcb485dd452696b4b11#diff-0f4f89ace79fa15278d9b283c2d9d9b2R84
>
>Then, by creating a key named `order`, containing the short names as a
>list.  Enabled plugins not listed will be executed *after* plugins
>listed, but in non-determined order.

The phrase "enabled plugins" implies that there can be disabled
plugins as well.



>For instance, consider the following entry points::
>
>  'avocado.plugins.result' : [
>     'xunit = avocado.plugins.xunit:XUnitResult',
>     'json = avocado.plugins.jsonresult:JSONResult',
>     'archive = avocado.plugins.archive:Archive',
>     'mail = avocado.plugins.mail:Mail',
>     'html = avocado_result_html:HTMLResult'
>   ]
>
>We can say that:
>
>* The plugin type, according to the fully qualified plugin name
>  definition here is `result`.
>
>* The plugin fully qualified names are:
>  - result.xunit
>  - result.json
>  - result.archive
>  - result.mail
>  - result.html
>
>* The short names for plugins of type "result" are:
>  - xunit
>  - json
>  - archive
>  - mail
>  - html

I like this use of examples. The illustrations are clear and easy to
understand.


>To make sure that the mail plugin is run after (and thus includes)
>the HTML result, the following configuration entry can be set::
>
>  [plugins.result]
>  order = html, archive

What does it mean for a plugin to "include" the results of an earlier
plugin?


>The other result plugins, namely xunit, json and mail, will still
>be run.  It's guaranteed they'll be run *after* the other result
>plugins.  The order in which they'll run after the explicitly
>ordered plugins is undefined.

Can a plugin determine what plugin(s) have run before it? (I don't
think it's necessary.)


>Other possible approach
>-----------------------
>
>The other approach possible, would require a default order value
>for plugins.  This would still preferably be done in configuration
>rather than in code.  Then, the fully qualified name for a plugin could
>be used as part of the configuration section.  Example::
>
>  [plugin.result.archive]
>  order = 50
>
>  [plugin.result.html]
>  order = 30

Small typo: "plugins.result" not "plugin.result" (two lines).


>This would make the `html` plugin run before the `archive` plugin.
>While more verbose, it would allow for external plugins to ship with
>stock configuration files that would set, by default, its ordering.

Order is here is used as a numerical value to indicate a relative
ordering, correct? I'm not sure I like the name "order"; how about
"sequence"?

A drawback to this approach is that you still have to come up with a
rule for what happens when two plugins have the same sequence number.

>Feedback is highly appreciated!

Another way to specify the order is to use an attribute at the
[plugins] or [plugins.result] level with certain expected values:

 [plugins]
 execution-order = random | lexical | user-defined

 where 'user-defined' would require yet another attribute
 that defines the sequence. 'user-defined' would also have to
 handle the 'unspecified' condition.
 
I think this is too complicated, but I offer it as a counter-example.

Finally, the typo got me to thinking: should it be "plugin" or
"plugins"? I don't care one way or the other. However, I do believe
that the attribute should be named consistent with the rest of
avocado. If there isn't a style guide for avocado proposals and naming
conventions, it would be good to have one. Consistency is going to be
hard to establish and maintain without it.

-Jeff


>
>-- 
>Cleber Rosa
>[ Sr Software Engineer - Virtualization Team - Red Hat ]
>[ Avocado Test Framework - avocado-framework.github.io ]
>