<div dir="ltr"> <div class="gmail_quote"><div dir="ltr">On Fri, Feb 10, 2017 at 11:16 AM Cleber Rosa <<a href="mailto:crosa@redhat.com">crosa@redhat.com</a>> wrote: </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> On 02/10/2017 05:01 AM, Andrei Stepanov wrote: > Hi. > > We need a reliable mechanism to notify Beaker server about start+result > of a avocado-vt test. As you know, avocado-vt test is one of the many > tests produced from cartesian config. We need to notify Beaker about > start+results of _each_ test. > > Currently we use: pre_command / post_command > > I discovered yesterday, that this commands are not called for FAILED > tests. Especially those have a error in syntax. > > In IRC Lukáš Doktor suggested to: > > 19:22<ldoktor>astepano: Hello Andrei, the `post_command` is part of the > `env_process` during the postprocess which means it's one of the cleanup > steps the problem is when the postprocess fails before getting to > `post_command` it will not be executed. It all depends on many factors. > Anyway I don't know what all information do you need to produce your > results but I'd strongly recommend writing either `JobPostTests` plugin, > or if you need per-test granularity t > 19:23<ldoktor>Also writing such plugin is really simple and there are > examples in our sources... > > So, I want to bring our conversation to this mail listing. > > I am not sure that such plugin will do job for avocado-vt tests. > Avacodo-vt tests generated from cartesian configs. > > Could you please suggest me the right approach to coupe with this issue? > > I need mechanism to call external program before & after _each_ > avocado-vt test. The program's environment should have variables: > TESTNAME / TESTRESULT. > Andrei, As a *very* brief answer, I'd say this looks like something that can be implemented as a `ResultEvents` plugin: * <a href="http://avocado-framework.readthedocs.io/en/45.0/ResultFormats.html#implementing-other-result-formats" rel="noreferrer" class="gmail_msg" target="_blank">http://avocado-framework.readthedocs.io/en/45.0/ResultFormats.html#implementing-other-result-formats</a> * <a href="http://avocado-framework.readthedocs.io/en/45.0/api/core/avocado.core.html#avocado.core.plugin_interfaces.ResultEvents" rel="noreferrer" class="gmail_msg" target="_blank">http://avocado-framework.readthedocs.io/en/45.0/api/core/avocado.core.html#avocado.core.plugin_interfaces.ResultEvents</a> You would write methods such as "start_test" and "end_test" to send the needed info to Beaker. This would be a feature generic to all Avocado supported tests (including Avocado-VT tests). </blockquote><div> </div><div>Nice. with a ResultEvents beaker plugin, any avocado test would be able to report results to a beaker server, and plugin configuration can be done using standard config file sections. +1.</div></div></div>