[Avocado-devel] pre_command / post_command clarification
Lukáš Doktor
ldoktor at redhat.com
Thu Mar 30 13:43:26 UTC 2017
Dne 27.3.2017 v 17:12 Radek Duda napsal(a):
> Hello Lukas,
>
> pro/post plugin does not solve this problem. We need to report to Beaker
> test name and result after *every* test run (and not after job - e.g.
> set of tests if I understand it correctly) (see Andrei's post
> https://www.redhat.com/archives/avocado-devel/2017-February/msg00043.html).
> pre/post plugin script is run only before/after job run. So is there any
> other way besides `ResultEvents` plugin (not available in avocado
> 36.0lts) to get result and name of each test?
>
Oh, I missed that requirement. Unfortunately that was introduced later.
Honestly I'd recommend using non-lts version instead, they are also
available in PIP and we are already talking about stabilizing before new
LTS release.
Anyway if you still want to do this in 36lts, you can write a pre
plugin, which would use `job.result_proxy.add_output_plugin()` to
register your custom plugin inherited from
`avocado.core.result.TestResult`. Basically this was the old way of
defining plugins before Cleber turned them to the proper Stevedore
plugins so it should work as long as you stay with 36lts, then you'd
simply adopt the code to be the proper `ResultEvents` plugin (with
similar structure).
Does this help?
Lukáš
PS: I'm tracing the master (not released, but really the master branch)
in my testing virt mini-CI and never had an issue with that.
>
> kind regards,
>
> Radek
>
> On Thu, Mar 16, 2017 at 11:56 AM, Lukáš Doktor <ldoktor at redhat.com
> <mailto:ldoktor at redhat.com>> wrote:
>
> Hello Radek,
>
> my proposal was to write a custom pre/post plugin, not just a
> pre/post script. You'd then ship your plugin to your test machines
> and install similarly to avocado-vt installation and if your plugin
> is generic enough, you can even propose it to avocado-vt upstream.
> There is already the `vt_joblock` pre/post plugin there
> (avocado-vt/avocado_vt/plugins/vt_joblock.py).
>
> I don't know of any way to obtain the information you need in
> `pre.d` script. In `post.d` script you could theoretically parse the
> `results` directory which you can get from the environment, but such
> solution would be quite ugly.
>
> Kind regards,
> Lukáš
>
> Dne 15.3.2017 v 15:41 Radek Duda napsal(a):
>
> Hi Lukas,
> concerning our IRC discussion (Lukas proposed to use
> job.test_suite[0][1].vt_param to obtain particular test
> parameters at
> the pre test phase):
> I still do not know how to solve accessibility of `job` variable
> in bash
> script (which should be created in dirs
> /etc/avocado/scripts/job/{pre.d,
> post.d}). The only solution seems to me is to export desired
> variables
> (test status, test name) to env (add it to
> `_job_to_environment_variables` method in `jobscripts.py`), but
> it is
> not good idea to mix avcado-vt variables to avocado code and I
> prefer to
> avoid editing avocado code at all.
>
> regards,
>
> Radek
>
> On Tue, Mar 14, 2017 at 6:17 PM, Lukáš Doktor
> <ldoktor at redhat.com <mailto:ldoktor at redhat.com>
> <mailto:ldoktor at redhat.com <mailto:ldoktor at redhat.com>>> wrote:
>
> Hello Radek,
>
> Dne 14.3.2017 v 12:14 Radek Duda napsal(a):
>
> Is there a way to implement this using Avocado 36.0lts?. I
> cannot find
> ResultEvent method in lts release nor
> plugin_interfaces.py file.
>
> The location is different, but the pre/post plugins were already
> supported in 36lts. The definition is in
> `avocado.plugins.base` and
> there is `avocado.plugins.jobscripts` which is an
> implementation of
> a pre/post plugin (which allows to execute custom scripts, which
> might actually be enough in your case, don't know).
>
> Anyway I don't know how are you executing the tests, but if
> you use
> Jenkins (or other automated way) I'd recommend adding those
> commands
> there. Anyway if you run them manually then shared
> configuration or
> even a plugin is the best approach.
>
> Lukáš
>
>
> regards,
>
> Radek Duda
>
> On Fri, Feb 10, 2017 at 9:19 PM, Lucas Meneghel Rodrigues
> <lookkas at gmail.com <mailto:lookkas at gmail.com>
> <mailto:lookkas at gmail.com <mailto:lookkas at gmail.com>>
> <mailto:lookkas at gmail.com <mailto:lookkas at gmail.com>
> <mailto:lookkas at gmail.com <mailto:lookkas at gmail.com>>>> wrote:
>
>
>
> On Fri, Feb 10, 2017 at 11:16 AM Cleber Rosa
> <crosa at redhat.com <mailto:crosa at redhat.com>
> <mailto:crosa at redhat.com <mailto:crosa at redhat.com>>
> <mailto:crosa at redhat.com <mailto:crosa at redhat.com>
> <mailto:crosa at redhat.com <mailto:crosa at redhat.com>>>> wrote:
>
>
>
> On 02/10/2017 05:01 AM, Andrei Stepanov wrote:
> > Hi.
> >
> > We need a reliable mechanism to notify Beaker
> server
> about start+result
> > of a avocado-vt test. As you know, avocado-vt
> test is
> one of the many
> > tests produced from cartesian config. We need to
> notify Beaker about
> > start+results of _each_ test.
> >
> > Currently we use: pre_command / post_command
> >
> > I discovered yesterday, that this commands are not
> called for FAILED
> > tests. Especially those have a error in syntax.
> >
> > In IRC Lukáš Doktor suggested to:
> >
> > 19:22<ldoktor>astepano: Hello Andrei, the
> `post_command` is part of the
> > `env_process` during the postprocess which
> means it's
> one of the cleanup
> > steps the problem is when the postprocess
> fails before
> getting to
> > `post_command` it will not be executed. It all
> depends
> on many factors.
> > Anyway I don't know what all information do
> you need
> to produce your
> > results but I'd strongly recommend writing either
> `JobPostTests` plugin,
> > or if you need per-test granularity t
> > 19:23<ldoktor>Also writing such plugin is really
> simple and there are
> > examples in our sources...
> >
> > So, I want to bring our conversation to this
> mail listing.
> >
> > I am not sure that such plugin will do job for
> avocado-vt tests.
> > Avacodo-vt tests generated from cartesian configs.
> >
> > Could you please suggest me the right approach to
> coupe with this issue?
> >
> > I need mechanism to call external program before &
> after _each_
> > avocado-vt test. The program's environment
> should have
> variables:
> > TESTNAME / TESTRESULT.
> >
>
> Andrei,
>
> As a *very* brief answer, I'd say this looks like
> something that
> can be
> implemented as a `ResultEvents` plugin:
>
> *
>
>
> http://avocado-framework.readthedocs.io/en/45.0/ResultFormats.html#implementing-other-result-formats
> <http://avocado-framework.readthedocs.io/en/45.0/ResultFormats.html#implementing-other-result-formats>
>
> <http://avocado-framework.readthedocs.io/en/45.0/ResultFormats.html#implementing-other-result-formats
> <http://avocado-framework.readthedocs.io/en/45.0/ResultFormats.html#implementing-other-result-formats>>
>
>
> <http://avocado-framework.readthedocs.io/en/45.0/ResultFormats.html#implementing-other-result-formats
> <http://avocado-framework.readthedocs.io/en/45.0/ResultFormats.html#implementing-other-result-formats>
>
> <http://avocado-framework.readthedocs.io/en/45.0/ResultFormats.html#implementing-other-result-formats
> <http://avocado-framework.readthedocs.io/en/45.0/ResultFormats.html#implementing-other-result-formats>>>
> *
>
>
> http://avocado-framework.readthedocs.io/en/45.0/api/core/avocado.core.html#avocado.core.plugin_interfaces.ResultEvents
> <http://avocado-framework.readthedocs.io/en/45.0/api/core/avocado.core.html#avocado.core.plugin_interfaces.ResultEvents>
>
> <http://avocado-framework.readthedocs.io/en/45.0/api/core/avocado.core.html#avocado.core.plugin_interfaces.ResultEvents
> <http://avocado-framework.readthedocs.io/en/45.0/api/core/avocado.core.html#avocado.core.plugin_interfaces.ResultEvents>>
>
>
> <http://avocado-framework.readthedocs.io/en/45.0/api/core/avocado.core.html#avocado.core.plugin_interfaces.ResultEvents
> <http://avocado-framework.readthedocs.io/en/45.0/api/core/avocado.core.html#avocado.core.plugin_interfaces.ResultEvents>
>
> <http://avocado-framework.readthedocs.io/en/45.0/api/core/avocado.core.html#avocado.core.plugin_interfaces.ResultEvents
> <http://avocado-framework.readthedocs.io/en/45.0/api/core/avocado.core.html#avocado.core.plugin_interfaces.ResultEvents>>>
>
> You would write methods such as "start_test" and
> "end_test" to
> send the
> needed info to Beaker.
>
> This would be a feature generic to all Avocado
> supported
> tests
> (including Avocado-VT tests).
>
>
> Nice. with a ResultEvents beaker plugin, any avocado
> test
> would be
> able to report results to a beaker server, and plugin
> configuration
> can be done using standard config file sections. +1.
>
>
>
>
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 502 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20170330/862f533b/attachment.sig>
More information about the Avocado-devel
mailing list