[Avocado-devel] option --output-check-record behavior

[Ademar Reis]

On Wed, Sep 14, 2016 at 06:54:31PM +0200, Lukáš Doktor wrote:
> Dne 9.9.2016 v 23:25 Lucas Meneghel Rodrigues napsal(a):
> > 
> > 
> > On Fri, Sep 9, 2016 at 8:14 AM Marcos E. Matsunaga
> > <Marcos.Matsunaga at oracle.com <mailto:Marcos.Matsunaga at oracle.com>> wrote:
> > 
> >     Hi guys,
> > 
> >     First of all, thanks again for your help. I really appreciate it.
> > 
> >     I found an interesting behavior. If I set loglevel=info in
> >     /etc/avocado/avocado.conf, it will not produce any content in
> >     stderr.expected and stdout.expected. If I set loglevel=debug, then
> >     it will work as it should. I don't mind running in debug mode, but I
> >     am not sure the behavior should be affected by loglevel.
> > 
> >     Anyway, the question I have is about using --output-check-record
> >     when multiplexing. I notice that the files stdout.expected and
> >     stderr.expected get overwritten on each variant. I will assume there
> >     is a way to save each of the variant results and then use them to
> >     check. The problem is that I went through the documentation and
> >     didn't find anything that talks about it.
> This is the expected behavior. The `--output-check-record` is a simple tool
> to allow checking simple tests like `cat /etc/fedora-release`, it was never
> meant for heavy stuff including multiplexer.

Not really.

--output-check-* should be fully compatible with the multiplexer.
What happens is that it was designed in a time when the concepts
of what a Test is where not very clear and it needs to be fixed
now. That is, we have a bug.

Following the definitions from the "Test ID RFC", I would say the
.data directory should be in the format
<Test-Name>[.Variant-ID].data. Which means the multiplexer should
work fine when combined with output-check: both -record and
-check.

Thanks.
   - Ademar

> Consider running the same test
> with a different file or with adjusted multiplex file (different number of
> variants, ...). What would be the expected results?
> 
> Anyway looking at your test, I'd implement it as two tests:
> 
> 1. start
> 2. stop
> 
> Looking something like this:
> 
> ```
> def start(...):
>     # start the xen machine with given attributes
> 
> def stop(...):
>     # stop the xen machine with given attributes
> 
> class StartTest(...):
>     def test(self):
>         start()
>     def tearDown(self):
>         stop()
> 
> class StopTest(...):
>     def setUp(self):
>         start()
>     def test(self):
>         stop()
> ```
> 
> Which would make sure to always cleanup after itself. Other solution would
> be to have start & stop as a single test, but having one test to start a
> machine and leaving it after the test is finished does not look nice to me.
> 
> 
> > 
> >     Thanks again.
> > 
> >     BTW, is the whole development team Brazilian?
> > 
> > No, we also have Lukas, from Czech republic, and also contributors in
> > China and India.
> Actually we have two core (Red Hat) people located in Czech republic and one
> in the USA a incrementally we get more and more contributors from all around
> the world.
> 
> > 
> > 
> > 
> >     Regards,
> 
> Regards,
> Lukáš
> 
> > 
> >     Marcos Eduardo Matsunaga
> > 
> >     Oracle USA
> >     Linux Engineering
> > 
> >     “The statements and opinions expressed here are my own and do not
> >     necessarily represent those of Oracle Corporation.”
> > 
> 




-- 
Ademar Reis
Red Hat

^[:wq!