[Avocado-devel] RFC: Plugin Management API
Lukáš Doktor
ldoktor at redhat.com
Wed May 11 13:29:39 UTC 2016
Hello Cleber
Dne 9.5.2016 v 17:12 Cleber Rosa napsal(a):
> Note: the same content on this message is available at:
>
> https://github.com/clebergnu/avocado/blob/rfc_plugin_api/docs/rfcs/plugin-management.rst
>
>
> Some users may find it easier to read with a prettier formatting.
> Problem statement
> =================
>
> This RFC is an offshoot from the "Avocado Job API" RFC. It aims to
> review the current extensibility architecture in Avocado, define
> common terminology and propose an API that will allow management and
> configuration of those extensible components.
>
> Again, the upcoming proposals, while hopefully useful in their own
> rights, are heavily influenced by the perceived needs of the "Job
> API". To put it simply, the Job API is also in the scope of the
> problem attempted to be solved here.
>
> Review
> ======
>
> During its development, Avocado has adopted many different extensibility
> techniques. This is common in many projects and even more common in
> projects built around languages that facilitate the loading of modules
> at runtime and `duck typing`_.
>
> On the bright side, Avocado has already got rid of much of the custom
> code for handling those extensions, by adopting a mature external
> library called `stevedore`_. Also on the bright side, there's now the
> formal definition of the interfaces expected to be available by users
> of those plugins.
>
> Terminology
> ===========
>
> It was noticed during recent discussions that there seems to be a lack
> of common understanding about what a Plugin is, how they're supposed
> to be implemented and consumed. This section attempts to settle the
> proposed terminology.
>
> Plugin type
> -----------
>
> A logical categorization of an extensible subsystem. In a music
> player application, valid plugin types may include:
>
> * Audio decoder
> * Lyrics fetcher
>
> In an application such as Avocado, valid plugin types may include:
>
> * Test runner
> * Result formatter
>
> In Avocado, a given Plugin type maps to one setuptools entrypoint
> namespace,
> and to a interface declared as Python `abstract base classes`_.
>
> Plugin patterns
> ---------------
>
> A "plugin" is a generic enough term. A "plugin type" doesn't adds much
> more than a label to an interface. So, it's also useful to define
> different usage patterns.
>
> Driver
> ~~~~~~
>
> When a single plugin is used at a given time.
>
> This usage pattern provides its user an abstract and usually
> simplified view of the controlled resource.
>
> The obvious examples are, of course, device drivers. On most, if not
> all, Operating Systems there will only be one driver implementation
> active to control a given device at a given time. That is true even
> when there may be multiple driver implementations for the same type of
> device.
>
> Using the music application example, for any given audio file, a
> single audio decoder "driver" is used. Using Avocado as an example,
> for any given test to be run, a single runner "driver" would be used.
>
> Extensions
> ~~~~~~~~~~
>
> When multiple plugins can be used simultaneously.
>
> This usage pattern allows multiple and independent extensions to
> act on on a given subsystem or resource or task.
>
> Using the music application example, there may be multiple lyrics
> fetcher extensions, one for each different lyrics database. Using
> Avocado as an example, multiple result formatter may generate reports in
> different file formats for a single job.
>
> Interface Declaration
> ---------------------
>
> The formal announcement of the interfaces that can be expected by
> users of a given plugin. Consequently an interface declaration defines
> what must be implemented by individual plugins.
>
> Plugin Management API
> ---------------------
>
> A set of functionalities that enable the application to load,
> unload, configure, activate and deactivate plugins.
>
> Current Plugin Review
> =====================
>
> Avocado relies exclusively on setuptools entrypoints to locate
> plugins. A setuptools entrypoint ties together the following 3
> pieces of information:
>
> 1. namespace, associated with a specific plugin type
> 2. plugin name, should be unique for a given namespace
> 3. reference to the plugin implementation, pointing all the way to the
> Python class
>
> One example of such an entrypoint declaration::
>
> 'avocado.plugins.cli': [
> 'gdb = avocado.plugins.gdb:GDB',
> ]
>
> Here we have:
>
> 1. ``avocado.plugins.cli``, the namespace for all plugins of the
> ``CLI`` type.
> 2. ``gdb`` as the name for this ``CLI`` plugin.
> 3. Implemented as the ``GDB`` class of the ``avocado.plugins.gdb``
> Python module.
>
> Avocado, by means of the various dispatcher classes, loads all
> registered plugins. Thus, if the plugin class can be loaded, it will
> be loaded by the current dispatcher at its initialization type.
>
> Finally, most dispatchers, at the appropriate time, will attempt to
> execute the implemented methods of all loaded plugins, by means of a
> ``map_method()`` call.
>
> The reason for that simplistic behavior is that all the current "new
> style" plugin are mapped to the "extensions" pattern. For instance,
> simply by adding registering a new plugin implementation at the
> ``avocado.plugins.cli`` namespace, that plugin method ``configure()``
> will be executed (which is supposed to add its arguments to the
> command line application).
>
> For other plugin types and usage patterns, the current activation
> method is not always the best choice. That is, even if a plugin
> can be loaded by Avocado and is ready to be used, it doesn't mean
> that it should be used.
>
> For instance, on plugins that map to the driver usage patterns, it
> does not make sense to load all plugins of a given type. The loading
> and activation should be deferred to the time that a choice is made on
> a specific implementation.
>
> For extension usage patterns, it should allow the user to add or
> remove any number of implementations for a given user or resource at
> runtime.
>
> Example
> =======
>
> Now, let's give a more concrete example of some of the concepts
> introduced earlier.
>
> The context used here is composed by the latest Avocado developments
> in the extensibility area, and also the proposed upcoming
> developments, with a focus on the "Job API" proposal.
>
> Interface Declaration
> ---------------------
>
> One of the recent additions to the "new plugin" standard in Avocado is
> the formal declaration of interfaces. When a new type of plugin is
> introduced, to make a specific subsystem of Avocado more modular, a
> formal interface is introduced. Then, when code is written to
> implement a specific Avocado subsystem, it MUST implement at least a
> minimum set of methods as properly defined in that interface.
>
> One example would be test runners. The declaration of an interface
> could be similar to::
>
> class Runner(Plugin):
> @abstractmethod # makes the implementation mandatory
> def run_test(self, test_factory):
>
> # noop to be inherited by implementations of this interface
> def setup(self):
> pass
>
> # noop to be inherited by implementations of this interface
> def tear_down(self):
> pass
>
> Please note that, while a single method (``run_test()``) is marked as
> mandatory by the ``@abstractmethod`` decorator, the overall interface
> is also composed by ``setup()`` and ``tear_down()``. Users of plugins
> that implement this interface should be able to safely call all
> interface methods, given that non-mandatory will have default
> implementations. These default implementations can either be "noops"
> or code that is sensible enough to be used by default by any plugin
> implementing the given interface.
>
> Interface Implementation
> ------------------------
>
> The implementation of a valid test runner could be similar to::
>
> class Container(Runner):
> def setup(self):
> prepare_container_image()
>
> def run_test(self, test_factory):
> reference = create_reference(test_factory)
> copy_to_image(reference)
> return run_from_image(reference)
>
> def tear_down(self):
> destroy_container_image()
>
> Also valid would be one that implements only the mandatory methods and
> inherit default implementations from the base class::
>
> class Local(Runner):
> def run_test(self, test_factory):
> test = create_test_from_factory(test_factory)
> return fork_and_run(test)
>
> Using an Interface
> ------------------
>
> Assuming that implementations of a given interface are correctly
> registered and loaded (more on that on the next section), users
> interested in that subsystem can use any implmentation in an agnostic
> way.
>
> As an example, code that implements the concept of a Job, can leverage
> any of the "Runner" implementations transparently. If a Job
> implementation wants to respect what the user somehow set as the
> default runner, it could implement something similar to::
>
> class Job(object):
> def __init__(self):
> default_runner = settings.get_value('runner', 'default',
> default='Local')
> self.runner = plugin_mgmt.new('avocado.plugins.runner',
> default_runner)
> def run(self):
> pre_tests_hooks()
> self.runner.setup()
> for test in tests:
> self.runner.run_test(test)
> self.runner.tear_down()
> post_test_hooks()
>
> Both example implementations should work similarly and transparently
> to the user (the Job implementation in this example). One of the
> proposed methods for the Plugin Management API has been quietly
> introduced in this example. It will be thoroughly discussed later.
>
> Plugin management API
> =====================
>
> On the previous section, the idea of a plugin management API was
> quietly and briefly planted::
>
> 1 class Job(object):
> 2 def __init__(self):
> 3 default_runner = settings.get_value('runner', 'default',
> 4 default='local')
> 5 self.runner = plugin_mgmt.new('avocado.plugins.runner',
> 6 default_runner)
>
so far so good.
> Lines 5 and 6, refer to a proposed method called ``new()`` of an also
> proposed module named ``plugin_mgmt``. Names are quite controversial,
> and not really the goal at this point, so please bear with the naming
> choices made so far, and feel free to suggest better ones.
>
> It should be clear that the goal of the ``new()`` method is to make
> an extensible subsystem implementation ready to be used. Its
> implementation, directly or indirectly, may involve locating the
> Python file that contains the associated class, loading that file into
> the current interpreter, creating a class instance, and finally,
> returning it.
>
> This maps well to the driver pattern, where little or no code is necessary
> around the plugin class instance itself.
>
> For usage patterns that map to the extensions definition given before, the
> "dispatcher" code may have higher level and additional methods::
>
> 01 class ResultFormatterDispatcher:
> 02
> 03 NAMESPACE = "avocado.plugin.result.format"
> 04
> 05 def add(self, name):
> 06 "Adds a plugin to the list of active result format writers"
> 07 self.active_set.add(plugin_mgmt.new(self.NAMESPACE, name)
> 08
> 09 def remove(self, name):
> 10 "Removes a plugin from the list of active result format
> writers"
> 11 self.active_set.remove_by_name(name)
> 12
> 13 def set_active(self, names):
> 14 "Adds or removes plugins so that only given plugin names
> are active"
> 15 ...
>
> Which could be used as::
>
> class Job(object):
> def __init__(self):
> ...
> self.result_formats = settings.get_value('result', 'formats',
> default=['json',
> 'xunit'])
> self.result_dispatcher = plugin_mgmt.ResultFormatterDispatcher()
> ...
>
> def run(self):
> self.result_dispatcher.set_active(self.result_formats)
> ...
> for test in tests:
> self.runner.run_test(test)
> ...
>
This would only work for a single-plugin-instance-environment. So for
`Cli` plugins that's fine. But this does not suit the current Results
plugins and it does not fits the possibility to get different plugins
within one execution (Job/Test API).
So we either need to modify result plugins to produce multiple outputs
and modify the framework/runner code to collect and forward all the
arguments to the single plugin instance, or we can allow the dispatcher
to add not plugins, but instances, using `dispatcher.add(plugin, *args,
**kwargs)`.
self.result_dispatcher = plugin_mgmt.new('avocado.plugins.result.'
'format')
self.result_dispatcher.add('xunit', result_dir + "/results.xml")
self.result_dispatcher.add('json', result_dir + "/results.json")
if args.get('json'):
# args.get(json) contains result dir of json plugin (--json)
self.result_dispatcher.add('json', args.get('json'))
The initialization could also report requirements, for example `--json
-` would report `["stdout"]` and if `result_dispatcher` receives
multiple requests for the same requirement, it should fail to register
the plugin.
Worth mention we should allow the dispatcher to initialize all available
plugins (with black list support) using default params, which would be
used by `Cli` plugins.
>
> Activation Scope
> ----------------
>
> It was mentioned during the definition of the different plugin patterns
> that only one driver plugin would be active at a given time. This is a
> simplification, one that doesn't take into account any kind of scopes.
>
> Avocado's code should implement contained scopes and add/remove
> plugins instances to these scopes only. For instance, on a single
> job, there may be multiple parallel test runners. The activation
> scope for a test runner driver plugin, is of course, individual to
> each runner.
>
If I understand this properly you're suggesting that the plugin
interface should allow several dispatchers of the same time to be
instantiated, containing different instances of the same plugins?
> Layered APIs
> ============
>
> It may be useful to provide more focused APIs as, supposedly, thin
> layers around the features provided by Plugin Management API. One
> example may be the activation and deactivation of test result
> formatters. Example::
>
> class Job(object):
> def add_runner(self, plugin_name):
> self.runners.append(plugin_mgmt.new('avocado.plugins.runner',
> plugin_name))
>
> This simplistic but quite real example has the goal of allowing users
> of the ``Job`` class to simply call::
>
> parallel_job = Job()
> for count in xrange(multiprocessing.cpu_count()):
> parallel_job.add_runner('local')
> ...
> job.run_parallel(test_list)
>
I'm not really sure how this is suppose to work. Can you please
elaborate a bit?
> Conclusion
> ==========
>
> Hopefully this text helps to pin-point the aspects of the Avocado
> architecture that, even though may need adjustments, can contribute to
> the implementation of the ultimate goal of providing a "Job API".
>
Yep, the introduction is awesome. I'm concerned about the
multiple-plugin-instances. We need a means to pass arguments to each
instance.
So IMO we have 3 types of plugins:
1. driver (allow just one instance at a time and scope)
2. extension (allow and usually uses one instance of each existing plugin)
3. proxy-like (allow several instances of any supported plugin, usually
mixed)
Examples would be:
1. CliCmd
2. Cli
3. TestResult
Where the 1 and 2 uses settings/args, but the 3 gets the arguments from
code. So we either need to adjust the plugins to allow multiple
arguments and pass the arguments from code to args, or to allow multiple
instances and arguments to dispatcher.
Lukáš
> .. _duck typing: https://en.wikipedia.org/wiki/Duck_typing
> .. _stevedore: https://pypi.python.org/pypi/stevedore
> .. _abstract base classes: https://docs.python.org/2/library/abc.html
>
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20160511/17692ab5/attachment.sig>
More information about the Avocado-devel
mailing list