[Avocado-devel] option --output-check-record behavior

[Ademar Reis]

On Wed, Sep 14, 2016 at 05:27:03PM -0300, Ademar Reis wrote:
> On Wed, Sep 14, 2016 at 06:54:31PM +0200, Lukáš Doktor wrote:
> > Dne 9.9.2016 v 23:25 Lucas Meneghel Rodrigues napsal(a):
> > > 
> > > 
> > > On Fri, Sep 9, 2016 at 8:14 AM Marcos E. Matsunaga
> > > <Marcos.Matsunaga at oracle.com <mailto:Marcos.Matsunaga at oracle.com>> wrote:
> > > 
> > >     Hi guys,
> > > 
> > >     First of all, thanks again for your help. I really appreciate it.
> > > 
> > >     I found an interesting behavior. If I set loglevel=info in
> > >     /etc/avocado/avocado.conf, it will not produce any content in
> > >     stderr.expected and stdout.expected. If I set loglevel=debug, then
> > >     it will work as it should. I don't mind running in debug mode, but I
> > >     am not sure the behavior should be affected by loglevel.
> > > 
> > >     Anyway, the question I have is about using --output-check-record
> > >     when multiplexing. I notice that the files stdout.expected and
> > >     stderr.expected get overwritten on each variant. I will assume there
> > >     is a way to save each of the variant results and then use them to
> > >     check. The problem is that I went through the documentation and
> > >     didn't find anything that talks about it.
> > This is the expected behavior. The `--output-check-record` is a simple tool
> > to allow checking simple tests like `cat /etc/fedora-release`, it was never
> > meant for heavy stuff including multiplexer.
> 
> Not really.
> 
> --output-check-* should be fully compatible with the multiplexer.
> What happens is that it was designed in a time when the concepts
> of what a Test is where not very clear and it needs to be fixed
> now. That is, we have a bug.
> 
> Following the definitions from the "Test ID RFC", I would say the
> .data directory should be in the format
> <Test-Name>[.Variant-ID].data. Which means the multiplexer should
> work fine when combined with output-check: both -record and
> -check.

https://trello.com/c/wiGkOFSa/828-test-s-data-directory-should-include-the-variant-id

Thanks.
   - Ademar

> 
> > Consider running the same test
> > with a different file or with adjusted multiplex file (different number of
> > variants, ...). What would be the expected results?
> > 
> > Anyway looking at your test, I'd implement it as two tests:
> > 
> > 1. start
> > 2. stop
> > 
> > Looking something like this:
> > 
> > ```
> > def start(...):
> >     # start the xen machine with given attributes
> > 
> > def stop(...):
> >     # stop the xen machine with given attributes
> > 
> > class StartTest(...):
> >     def test(self):
> >         start()
> >     def tearDown(self):
> >         stop()
> > 
> > class StopTest(...):
> >     def setUp(self):
> >         start()
> >     def test(self):
> >         stop()
> > ```
> > 
> > Which would make sure to always cleanup after itself. Other solution would
> > be to have start & stop as a single test, but having one test to start a
> > machine and leaving it after the test is finished does not look nice to me.
> > 
> > 
> > > 
> > >     Thanks again.
> > > 
> > >     BTW, is the whole development team Brazilian?
> > > 
> > > No, we also have Lukas, from Czech republic, and also contributors in
> > > China and India.
> > Actually we have two core (Red Hat) people located in Czech republic and one
> > in the USA a incrementally we get more and more contributors from all around
> > the world.
> > 
> > > 
> > > 
> > > 
> > >     Regards,
> > 
> > Regards,
> > Lukáš
> > 
> > > 
> > >     Marcos Eduardo Matsunaga
> > > 
> > >     Oracle USA
> > >     Linux Engineering
> > > 
> > >     “The statements and opinions expressed here are my own and do not
> > >     necessarily represent those of Oracle Corporation.”
> > > 
> > 
> 
> 
> 
> 
> -- 
> Ademar Reis
> Red Hat
> 
> ^[:wq!

-- 
Ademar Reis
Red Hat

^[:wq!