[Avocado-devel] pre_command / post_command clarification
Radek Duda
rduda at redhat.com
Mon Mar 27 15:12:18 UTC 2017
Hello Lukas,
pro/post plugin does not solve this problem. We need to report to Beaker
test name and result after *every* test run (and not after job - e.g. set
of tests if I understand it correctly) (see Andrei's post
https://www.redhat.com/archives/avocado-devel/2017-February/msg00043.html).
pre/post plugin script is run only before/after job run. So is there any
other way besides `ResultEvents` plugin (not available in avocado 36.0lts)
to get result and name of each test?
kind regards,
Radek
On Thu, Mar 16, 2017 at 11:56 AM, Lukáš Doktor <ldoktor at redhat.com> wrote:
> Hello Radek,
>
> my proposal was to write a custom pre/post plugin, not just a pre/post
> script. You'd then ship your plugin to your test machines and install
> similarly to avocado-vt installation and if your plugin is generic enough,
> you can even propose it to avocado-vt upstream. There is already the
> `vt_joblock` pre/post plugin there (avocado-vt/avocado_vt/plugins
> /vt_joblock.py).
>
> I don't know of any way to obtain the information you need in `pre.d`
> script. In `post.d` script you could theoretically parse the `results`
> directory which you can get from the environment, but such solution would
> be quite ugly.
>
> Kind regards,
> Lukáš
>
> Dne 15.3.2017 v 15:41 Radek Duda napsal(a):
>
>> Hi Lukas,
>> concerning our IRC discussion (Lukas proposed to use
>> job.test_suite[0][1].vt_param to obtain particular test parameters at
>> the pre test phase):
>> I still do not know how to solve accessibility of `job` variable in bash
>> script (which should be created in dirs /etc/avocado/scripts/job/{pre.d,
>> post.d}). The only solution seems to me is to export desired variables
>> (test status, test name) to env (add it to
>> `_job_to_environment_variables` method in `jobscripts.py`), but it is
>> not good idea to mix avcado-vt variables to avocado code and I prefer to
>> avoid editing avocado code at all.
>>
>> regards,
>>
>> Radek
>>
>> On Tue, Mar 14, 2017 at 6:17 PM, Lukáš Doktor <ldoktor at redhat.com
>> <mailto:ldoktor at redhat.com>> wrote:
>>
>> Hello Radek,
>>
>> Dne 14.3.2017 v 12:14 Radek Duda napsal(a):
>>
>> Is there a way to implement this using Avocado 36.0lts?. I
>> cannot find
>> ResultEvent method in lts release nor plugin_interfaces.py file.
>>
>> The location is different, but the pre/post plugins were already
>> supported in 36lts. The definition is in `avocado.plugins.base` and
>> there is `avocado.plugins.jobscripts` which is an implementation of
>> a pre/post plugin (which allows to execute custom scripts, which
>> might actually be enough in your case, don't know).
>>
>> Anyway I don't know how are you executing the tests, but if you use
>> Jenkins (or other automated way) I'd recommend adding those commands
>> there. Anyway if you run them manually then shared configuration or
>> even a plugin is the best approach.
>>
>> Lukáš
>>
>>
>> regards,
>>
>> Radek Duda
>>
>> On Fri, Feb 10, 2017 at 9:19 PM, Lucas Meneghel Rodrigues
>> <lookkas at gmail.com <mailto:lookkas at gmail.com>
>> <mailto:lookkas at gmail.com <mailto:lookkas at gmail.com>>> wrote:
>>
>>
>>
>> On Fri, Feb 10, 2017 at 11:16 AM Cleber Rosa
>> <crosa at redhat.com <mailto:crosa at redhat.com>
>> <mailto:crosa at redhat.com <mailto:crosa at redhat.com>>> wrote:
>>
>>
>>
>> On 02/10/2017 05:01 AM, Andrei Stepanov wrote:
>> > Hi.
>> >
>> > We need a reliable mechanism to notify Beaker server
>> about start+result
>> > of a avocado-vt test. As you know, avocado-vt test is
>> one of the many
>> > tests produced from cartesian config. We need to
>> notify Beaker about
>> > start+results of _each_ test.
>> >
>> > Currently we use: pre_command / post_command
>> >
>> > I discovered yesterday, that this commands are not
>> called for FAILED
>> > tests. Especially those have a error in syntax.
>> >
>> > In IRC Lukáš Doktor suggested to:
>> >
>> > 19:22<ldoktor>astepano: Hello Andrei, the
>> `post_command` is part of the
>> > `env_process` during the postprocess which means it's
>> one of the cleanup
>> > steps the problem is when the postprocess fails before
>> getting to
>> > `post_command` it will not be executed. It all depends
>> on many factors.
>> > Anyway I don't know what all information do you need
>> to produce your
>> > results but I'd strongly recommend writing either
>> `JobPostTests` plugin,
>> > or if you need per-test granularity t
>> > 19:23<ldoktor>Also writing such plugin is really
>> simple and there are
>> > examples in our sources...
>> >
>> > So, I want to bring our conversation to this mail
>> listing.
>> >
>> > I am not sure that such plugin will do job for
>> avocado-vt tests.
>> > Avacodo-vt tests generated from cartesian configs.
>> >
>> > Could you please suggest me the right approach to
>> coupe with this issue?
>> >
>> > I need mechanism to call external program before &
>> after _each_
>> > avocado-vt test. The program's environment should have
>> variables:
>> > TESTNAME / TESTRESULT.
>> >
>>
>> Andrei,
>>
>> As a *very* brief answer, I'd say this looks like
>> something that
>> can be
>> implemented as a `ResultEvents` plugin:
>>
>> *
>>
>> http://avocado-framework.readthedocs.io/en/45.0/ResultFormat
>> s.html#implementing-other-result-formats
>> <http://avocado-framework.readthedocs.io/en/45.0/ResultForma
>> ts.html#implementing-other-result-formats>
>>
>> <http://avocado-framework.readthedocs.io/en/45.0/ResultForma
>> ts.html#implementing-other-result-formats
>> <http://avocado-framework.readthedocs.io/en/45.0/ResultForma
>> ts.html#implementing-other-result-formats>>
>> *
>>
>> http://avocado-framework.readthedocs.io/en/45.0/api/core/
>> avocado.core.html#avocado.core.plugin_interfaces.ResultEvents
>> <http://avocado-framework.readthedocs.io/en/45.0/api/core/
>> avocado.core.html#avocado.core.plugin_interfaces.ResultEvents>
>>
>> <http://avocado-framework.readthedocs.io/en/45.0/api/core/
>> avocado.core.html#avocado.core.plugin_interfaces.ResultEvents
>> <http://avocado-framework.readthedocs.io/en/45.0/api/core/
>> avocado.core.html#avocado.core.plugin_interfaces.ResultEvents>>
>>
>> You would write methods such as "start_test" and
>> "end_test" to
>> send the
>> needed info to Beaker.
>>
>> This would be a feature generic to all Avocado supported
>> tests
>> (including Avocado-VT tests).
>>
>>
>> Nice. with a ResultEvents beaker plugin, any avocado test
>> would be
>> able to report results to a beaker server, and plugin
>> configuration
>> can be done using standard config file sections. +1.
>>
>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20170327/989acc6a/attachment.htm>
More information about the Avocado-devel
mailing list