[Avocado-devel] option --output-check-record behavior
Amador Segundo
asegundo at redhat.com
Thu Sep 8 18:39:57 UTC 2016
On Sep 8, 2016 8:19 PM, "Marcos E. Matsunaga" <Marcos.Matsunaga at oracle.com>
wrote:
>
> Hi Cleber,
>
> Sorry for being a pain. I've tried to use process.system to execute
commands, but I got an error, that's why I kept the autotest libraries (I
actually copied from one of the files in io-github-autotest-libvirt).
>
> The error I'm getting is:
>
> 2016-09-08 13:49:25,844 sysinfo L0371 INFO | Profiler disabled
> 2016-09-08 13:49:25,845 test L0214 INFO | START
1-/root/avocado-vt/io-fs-autotest-xen/xen/tests/xentest.py:xentest.test_xen_start_stop;1
> 2016-09-08 13:49:25,846 process L0334 INFO | Running 'xl
info|grep xen_version| awk '{print $3}''
> 2016-09-08 13:49:25,863 process L0435 INFO | Command 'xl
info|grep xen_version| awk '{print $3}'' finished with 1 after
0.00377798080444s
The problem here is the pipe. You can collect the output and parse it
inside the python code. Something.like:
output = process.system_output("xl info")
for line in output.splitlines():
if 'xen_version' in line:
var = line.split()[3]
break
...
> 2016-09-08 13:49:25,863 stacktrace L0038 ERROR|
> 2016-09-08 13:49:25,863 stacktrace L0041 ERROR| Reproduced
traceback from: /usr/lib/python2.7/site-packages/avocado/core/test.py:448
> 2016-09-08 13:49:25,864 stacktrace L0044 ERROR| Traceback (most
recent call last):
> 2016-09-08 13:49:25,864 stacktrace L0044 ERROR| File
"/root/avocado-vt/io-fs-autotest-xen/xen/tests/xentest.py", line 52, in
test_xen_start_stop
> 2016-09-08 13:49:25,864 stacktrace L0044 ERROR| xenversion =
process.system_output(cmd, ignore_status=False)
> 2016-09-08 13:49:25,864 stacktrace L0044 ERROR| File
"/usr/lib/python2.7/site-packages/avocado/utils/process.py", line 1139, in
system_output
> 2016-09-08 13:49:25,864 stacktrace L0044 ERROR| sudo=sudo)
> 2016-09-08 13:49:25,864 stacktrace L0044 ERROR| File
"/usr/lib/python2.7/site-packages/avocado/utils/process.py", line 1046, in
run
> 2016-09-08 13:49:25,864 stacktrace L0044 ERROR| raise
CmdError(cmd, sp.result)
> 2016-09-08 13:49:25,864 stacktrace L0044 ERROR| CmdError: Command
'xl info|grep xen_version| awk '{print $3}'' failed (rc=1)
> 2016-09-08 13:49:25,864 stacktrace L0045 ERROR|
> 2016-09-08 13:49:25,865 test L0563 ERROR| Traceback (most
recent call last):
>
> 2016-09-08 13:49:25,865 test L0563 ERROR| File
"/usr/lib/python2.7/site-packages/avocado/core/test.py", line 503, in
_run_avocado
> raise test_exception
>
> 2016-09-08 13:49:25,865 test L0563 ERROR| CmdError: Command
'xl info|grep xen_version| awk '{print $3}'' failed (rc=1)
>
> I looked at "/usr/lib/python2.7/site-packages/avocado/utils/process.py",
line 1139 and didn't see anything that could be causing the problem. I even
tried to add "sudo=False" argument in process.system, but I get the same
error.
>
> Thanks again.
>
>
> On 09/08/2016 06:36 PM, Cleber Rosa wrote:
>>
>> On 09/08/2016 01:50 PM, Marcos E. Matsunaga wrote:
>>>
>>> On 09/08/2016 05:44 PM, Cleber Rosa wrote:
>>>>
>>>> On 09/08/2016 11:34 AM, Marcos E. Matsunaga wrote:
>>>>>
>>>>> Hi Cleber,
>>>>>
>>>>> Thanks for your quick reply. That's exactly what I understood, but
here
>>>>> is what is happening
>>>>>
>>>>> I have a directory ~/avocado/xen/tests where I have the xentest.py
>>>>> script. When I execute it, it does create the directory
>>>>> ~/avocado/xen/tests/xentest.py.data with stderr.expected and
>>>>> stdout.expected (empty). It also creates the two files (stdout and
>>>>> stderr) in the job-results/latest directory, but also empty.
>>>>>
>>>>> The weird thing is that instead of saving, it reports to the job.log
as
>>>>> an error "L0151 ERROR| [stderr] Parsing config from /VM/guest1/vm.cf".
>>>>>
>>>>> That's why I think I am missing something.
>>>>
>>>> Can you post the full test code and the resulting `job.log` file?
>>>
>>> Sure.. It is attached.
>>> And the multiplex file I am using is:
>>>
>>> xentest:
>>> guest1:
>>> action: !mux
>>> start:
>>> run_action: "create"
>>> domain_name: "perf1"
>>> sleep_time: 1
>>> stop:
>>> run_action: "shutdown"
>>> domain_name: "perf1"
>>> sleep_time: 60
>>> guest_cfg: /Repo/VM/perf1/vm.cfg
>>>
>>>>> Thanks again for your help.
>>>>>
>>>>> On 09/08/2016 02:59 PM, Cleber Rosa wrote:
>>>>>>
>>>>>> On 09/08/2016 10:25 AM, Marcos E. Matsunaga wrote:
>>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> I am new to avocado and have just started to look into it.
>>>>>>>
>>>>>>> I have been playing with avocado on Fedora 24 for a few weeks. I
>>>>>>> wrote a
>>>>>>> small script to run commands and was exploring the option
>>>>>>> "--output-check-record", but it never populate the files
>>>>>>> stderr.expected
>>>>>>> and stdout.expected. Instead, it prints an error with "[stderr]" in
>>>>>>> the
>>>>>>> job.log file. My understanding is that the output (stderr and
stdout)
>>>>>>> of commands/scripts executed by avocado would be captured and saved
on
>>>>>>> those files (like on synctest.py example), but it doesn't. I want to
>>>>>>> know if I am doing something wrong or it is a bug.
>>>>>>>
>>>>>> Hi Marcos,
>>>>>>
>>>>>> Avocado creates the `stdout` and `stderr` files in the test result
>>>>>> directory. In the synctest example, for instance, my contains:
>>>>>>
>>>>>> $ avocado run examples/tests/synctest.py
>>>>>> $ cat
>>>>>>
~/avocado/job-results/latest/test-results/1-examples_tests_synctest.py\:SyncTest.test/stdout
>>>>>>
>>>>>>
>>>>>>
>>>>>> PAR : waiting
>>>>>> PASS : sync interrupted
>>>>>>
>>>>>> `stderr` is actually empty for that test:
>>>>>>
>>>>>> $ wc -l
>>>>>>
~/avocado/job-results/latest/test-results/1-examples_tests_synctest.py\:SyncTest.test/stderr
>>>>>>
>>>>>>
>>>>>> 0
>>>>>>
/home/cleber/avocado/job-results/latest/test-results/1-examples_tests_synctest.py:SyncTest.test/stderr
>>>>>>
>>>>>>
>>>>>>
>>>>>> What you have to do is, once you're satisfied with those outputs, and
>>>>>> they're considered "the gold standard", you'd move those to the test
>>>>>> *data directory*.
>>>>>>
>>>>>> So, if you test is hosted at, `/tests/xl.py`, you'd created the
>>>>>> `/tests/xl.py.data`, and put those files there, named
`stdout.expected`
>>>>>> and `stderr.expected`.
>>>>>>
>>>>>> Whenever you run `avocado run --output-check-record all
/tests/xl.py`,
>>>>>> those files will be used and the output of the *current* test
execution
>>>>>> will be compared to those "gold standards".
>>>>>>
>>>>>>> The script is very simple and the way I execute the command is:
>>>>>>>
>>>>>>> cmd = ('/usr/sbin/xl create /VM/guest1/vm.cfg')
>>>>>>> if utils.system(cmd) == "0":
>>
>> The issue seems to be related to the fact that you're using old autotest
>> libraries to execute your external commands.
>>
>> The output record/check support is built into Avocado's libraries,
>> namely `avocado.utils.process`.
>>
>> Try to replace your code with:
>>
>> from avocado.utils import process
>> ...
>> process.system(cmd)
>>
>> Also, mixing and matching autotest and avocado libraries in the same
>> test code is not something I'd recommend. Not that it's guaranteed to
>> break. I'd recomend to (try to) move to avocado libraries, or keep an
>> eye on strange behavior like this.
>>
>>>>>>> pass
>>>>>>> else:
>>>>>>> return False
>>>>>>>
>>>>>>> The command send to stdout:
>>>>>>>
>>>>>>> Parsing config from /VM/guest1/vm.cfg
>>>>>>>
>>>>>>> I run the test as:
>>>>>>>
>>>>>>> avocado run --output-check-record all xentest.py
>>>>>>>
>>>>>>> The job.log file contains:
>>>>>>>
>>>>>>> 2016-09-07 13:04:48,015 test L0214 INFO | START
>>>>>>>
1-/root/avocado-vt/io-fs-autotest-xen/xen/tests/xentest.py:xentest.test_xen_start_stop;1
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2016-09-07 13:04:48,051 xentest L0033 INFO |
>>>>>>>
1-/root/avocado-vt/io-fs-autotest-xen/xen/tests/xentest.py:xentest.test_xen_start_stop;1:
>>>>>>>
>>>>>>>
>>>>>>> Running action create
>>>>>>> 2016-09-07 13:04:49,067 utils L0151 ERROR| [stderr]
Parsing
>>>>>>> config from /VM/guest1/vm.cfg
>>>>>>> 2016-09-07 13:04:49,523 test L0586 INFO | PASS
>>>>>>>
1-/root/avocado-vt/io-fs-autotest-xen/xen/tests/xentest.py:xentest.test_xen_start_stop;1
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks for your time and help.
>>>>>>
>>>>>> Let me know if it's clear now! And thanks for trying Avocado out!
>>>>>>
>
> --
>
> Regards,
>
> Marcos Eduardo Matsunaga
>
> Oracle USA
> Linux Engineering
>
> “The statements and opinions expressed here are my own and do not
> necessarily represent those of Oracle Corporation.”
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20160908/6fadfbaf/attachment.htm>
More information about the Avocado-devel
mailing list