[Avocado-devel] Multiplex configuration failing
Olav Philipp Henschel
olavph at linux.vnet.ibm.com
Mon Nov 30 13:16:31 UTC 2015
I've tried to update PyYAML, but it is already at latest version. It
still does not work unless I remove the cyaml files.
The only difference in my system is a more up-to-date kernel.
kernel-3.10.82-2042.1.pkvm2_1_1.71.src.rpm
python-2.7.5-15.1.pkvm2_1_1.1.src.rpm
PyYaml 3.11
Regards,
Olav
On 28-11-2015 15:16, Lukáš Doktor wrote:
> Dear Olav,
>
> today I tried to reproduce it to fill the bugzilla, but everything
> started to work properly even on PKVM:
>
> kernel-3.10.53-2023.1.pkvm2_1_1.52.ppc64
> python-2.7.5-15.1.pkvm2_1_1.1.ppc64
> PyYAML (3.11)
>
> So probably you can just update your system (pip install --upgrade
> PyYAML) and it might start working without the workaround ...
>
> Regards,
> Lukáš
>
>
> Dne 25.11.2015 v 13:22 Olav Philipp Henschel napsal(a):
>> I did not report it, you can do it.
>>
>> On 25-11-2015 10:13, Lukáš Doktor wrote:
>>> Dne 19.11.2015 v 18:05 Lucas Meneghel Rodrigues napsal(a):
>>>> OK, the mystery is finally solved :)
>>>>
>>>> Lukas, what about we put a conditional that probes for that type of
>>>> machine and loads the pure python YAML loader instead of the CLoader?
>>>
>>> Well I thought about using `try/except` to catch the `ReaderError` and
>>> log information that maybe disabling CLoader could help, but then I
>>> said to myself that people should use ML archives to find temporary
>>> workarounds. We can't be adding notes about all kinds of problems. It
>>> works fine on RHEL, so hopefully IBM will fix it soon.
>>>
>>> Olav, did you reported the bug, or should I do it?
>>>
>>> Lukáš
>>>
>>>>
>>>> On one hand, it doesn't require from our users workarounds. On the
>>>> other
>>>> hand, when this bug gets fixed we'll never know and users won't get
>>>> the
>>>> benefit of using the C loader (which shouldn't be that big a deal for
>>>> most reasonably sized YAML files).
>>>>
>>>>
>>>> On Thu, Nov 19, 2015 at 11:44 AM Olav Philipp Henschel
>>>> <olavph at linux.vnet.ibm.com <mailto:olavph at linux.vnet.ibm.com>> wrote:
>>>>
>>>> Thanks, I've tried both solutions and they worked.
>>>> I'll just rename files
>>>> /usr/lib64/python2.7/site-packages/yaml/cyaml.py*
>>>> so they are not found.
>>>>
>>>> Regards,
>>>> Olav P. Henschel
>>>>
>>>> On 19-11-2015 11:24, Lukáš Doktor wrote:
>>>> > I tried the ppc64 BE with the same results, everything worked
>>>> well.
>>>> > Then I tried PowerKVM and finally I was able to reproduce it.
>>>> >
>>>> > The original exception is:
>>>> >
>>>> > ```
>>>> > Traceback (most recent call last):
>>>> > File "./scripts/avocado", line 85, in <module>
>>>> > sys.exit(app.run())
>>>> > File "/tmp/avocado/avocado/core/app.py", line 61, in run
>>>> > return self.parser.take_action()
>>>> > File "/tmp/avocado/avocado/core/parser.py", line 100, in
>>>> take_action
>>>> > return self.args.dispatch(self.args)
>>>> > File "/tmp/avocado/avocado/core/plugins/multiplexer.py",
>>>> line 100,
>>>> > in run
>>>> > args.debug)
>>>> > File "/tmp/avocado/avocado/core/multiplexer.py", line 99, in
>>>> yaml2tree
>>>> > input_tree = tree.create_from_yaml(input_yamls, debug)
>>>> > File "/tmp/avocado/avocado/core/tree.py", line 463, in
>>>> create_from_yaml
>>>> > merge(data, path)
>>>> > File "/tmp/avocado/avocado/core/tree.py", line 447, in _merge
>>>> > data.merge(_create_from_yaml(path))
>>>> > File "/tmp/avocado/avocado/core/tree.py", line 424, in
>>>> > _create_from_yaml
>>>> > loaded_tree = yaml.load(stream, Loader)
>>>> > File "/usr/lib64/python2.7/site-packages/yaml/__init__.py",
>>>> line 71,
>>>> > in load
>>>> > return loader.get_single_data()
>>>> > File
>>>> "/usr/lib64/python2.7/site-packages/yaml/constructor.py", line
>>>> > 37, in get_single_data
>>>> > node = self.get_single_node()
>>>> > File "_yaml.pyx", line 702, in _yaml.CParser.get_single_node
>>>> > (ext/_yaml.c:7647)
>>>> > File "_yaml.pyx", line 905, in
>>>> _yaml.CParser._parse_next_event
>>>> > (ext/_yaml.c:10396)
>>>> > ReaderError: unacceptable character #x0007: control characters
>>>> are not
>>>> > allowed
>>>> > in "examples/mux-environment.yaml", position 484
>>>> > ```
>>>> >
>>>> > Then I tried disabling the CLoader (PyYAML allows you to use
>>>> python or
>>>> > C version of the yaml parser) and it worked well.
>>>> >
>>>> > So you can either wait when python-2.7.5-34 is shipped (and
>>>> hope
>>>> it'll
>>>> > just start working), or you can modify
>>>> `avocado/core/tree.py` to
>>>> > import `Loader` instead of `from yaml import CLoader as
>>>> Loader`.
>>>> > Alternatively you can remove the
>>>> > /usr/lib/python2.7/site-packages/yaml/cyaml.py*` which would
>>>> result in
>>>> > yaml being imported instead.
>>>> >
>>>> > Although it doesn't seems to be a problem of `PyYAML` itself,
>>>> because
>>>> > I was using the latest copy&pasted version from my Fedora (I
>>>> didn't
>>>> > wanted to install packages on the pkvm host).
>>>> >
>>>> > I hope this will help,
>>>> > Lukáš
>>>> >
>>>> > Dne 19.11.2015 v 12:54 Olav Philipp Henschel napsal(a):
>>>> >> Thanks for the responses.
>>>> >> I am trying to run the selftests, but I'm having problems with
>>>> >> dependencies (pillow requires jpeg, which is not found in my
>>>> repos).
>>>> >> I'll try to solve that and will also try to run in a different
>>>> >> machine/distro. I am using a PowerKVM host.
>>>> >>
>>>> >> $ uname -a
>>>> >> Linux ultraseven 3.10.82-2042.1.pkvm2_1_1.71.ppc64 #1 SMP Fri
>>>> Jul 31
>>>> >> 09:52:38 CDT 2015 ppc64 ppc64 ppc64 GNU/Linux
>>>> >> $ rpm -q python
>>>> >> python-2.7.5-15.1.pkvm2_1_1.1.ppc64
>>>> >>
>>>> >>
>>>> >> On 19-11-2015 04:03, Lukáš Doktor wrote:
>>>> >>> Dne 19.11.2015 v 04:25 Lucas Meneghel Rodrigues napsal(a):
>>>> >>>>
>>>> >>>>
>>>> >>>> On Wed, Nov 18, 2015 at 5:24 PM Olav Philipp Henschel
>>>> >>>> <olavph at linux.vnet.ibm.com
>>>> <mailto:olavph at linux.vnet.ibm.com>
>>>> <mailto:olavph at linux.vnet.ibm.com
>>>> <mailto:olavph at linux.vnet.ibm.com>>> wrote:
>>>> >>>>
>>>> >>>> Thanks, Cleber, I thinks that clarifies the doc, but
>>>> that's not my
>>>> >>>> issue.
>>>> >>>> I've executed the same command (./avocado multiplex
>>>> --tree
>>>> >>>> ../examples/mux-environment.yaml) in another machine
>>>> (x86_64)
>>>> >>>> and it
>>>> >>>> worked.
>>>> >>>> It might be a problem with ppc64 architecture.
>>>> >>>>
>>>> >>>>
>>>> >>>> Now that's interesting. Let's see what Lukas has to say on
>>>> that
>>>> >>>> matter.
>>>> >>>>
>>>> >>>> One thing worth checking would be to run the avocado
>>>> regression
>>>> >>>> suite on
>>>> >>>> a ppc64 machine - it runs multiplex functionality tests as
>>>> well.
>>>> >>>> Please
>>>> >>>> try to:
>>>> >>>>
>>>> >>>> 1) Uninstall avocado rpms/debs
>>>> >>>> 2) clone avocado to some dir
>>>> >>>> 3) Certify that the selftests dependencies are installed -
>>>> >>>> see requirements-selftests.txt
>>>> >>>> 4) 'make check'
>>>> >>>>
>>>> >>>> I'm very interested in seeing what the functional suite
>>>> reports on
>>>> >>>> your
>>>> >>>> ppc64 test systems.
>>>> >>>
>>>> >>> Dear Olav,
>>>> >>>
>>>> >>> I haven't read everything yet, but I tried following:
>>>> >>>
>>>> >>> [root at ibm-p8-virt-01 tmp]# avocado multiplex --tree
>>>> >>> mux-environment.yaml
>>>> >>> ┗━━ run
>>>> >>> ┣━━ hw
>>>> >>> ┃ ┣━━ cpu
>>>> >>> ┃ ┃ ╠══ intel
>>>> >>> ┃ ┃ ╠══ amd
>>>> >>> ┃ ┃ ╚══ arm
>>>> >>> ┃ ┗━━ disk
>>>> >>> ┃ ╠══ scsi
>>>> >>> ┃ ╚══ virtio
>>>> >>> ┣━━ distro
>>>> >>> ┃ ╠══ fedora
>>>> >>> ┃ ╚══ mint
>>>> >>> ┗━━ env
>>>> >>> ╠══ debug
>>>> >>> ╚══ prod
>>>> >>>
>>>> >>> [root at ibm-p8-virt-01 tmp]# avocado multiplex --tree
>>>> >>> mux-environment.yaml -c
>>>> >>> ┗━━ run
>>>> >>> ┣━━ hw
>>>> >>> ┃ ┣━━ cpu
>>>> >>> ┃ ┃ ╠══ intel
>>>> >>> ┃ ┃ ║ → cpu_CFLAGS: -march=core2
>>>> >>> ┃ ┃ ╠══ amd
>>>> >>> ┃ ┃ ║ → cpu_CFLAGS: -march=athlon64
>>>> >>> ┃ ┃ ╚══ arm
>>>> >>> ┃ ┃ → cpu_CFLAGS: -mabi=apcs-gnu
>>>> -march=armv8-a
>>>> >>> -mtune=arm8
>>>> >>> ┃ ┗━━ disk
>>>> >>> ┃ ╠══ scsi
>>>> >>> ┃ ║ → disk_type: scsi
>>>> >>> ┃ ╚══ virtio
>>>> >>> ┃ → disk_type: virtio
>>>> >>> ┣━━ distro
>>>> >>> ┃ ╠══ fedora
>>>> >>> ┃ ║ → init: systemd
>>>> >>> ┃ ╚══ mint
>>>> >>> ┃ → init: systemv
>>>> >>> ┗━━ env
>>>> >>> ╠══ debug
>>>> >>> ║ → opt_CFLAGS: -O0 -g
>>>> >>> ╚══ prod
>>>> >>> → opt_CFLAGS: -O2
>>>> >>> [root at ibm-p8-virt-01 tmp]# uname -a
>>>> >>> Linux ibm-p8-virt-01.lab.bos.redhat.com
>>>> <http://ibm-p8-virt-01.lab.bos.redhat.com> 3.10.0-327.el7.ppc64le
>>>> #1 SMP
>>>> >>> Thu Oct 29 17:31:13 EDT 2015 ppc64le ppc64le ppc64le
>>>> GNU/Linux
>>>> >>> [root at ibm-p8-virt-01 tmp]# rpm -q python
>>>> >>> python-2.7.5-34.el7.ppc64le
>>>> >>>
>>>> >>>
>>>> >>> I'll be offline for few hours, but then I'll read the email
>>>> carefully
>>>> >>> and try to find why is it not working for you.
>>>> >>>
>>>> >>> Kind regards,
>>>> >>> Lukáš
>>>> >>>
>>>> >>> _______________________________________________
>>>> >>> Avocado-devel mailing list
>>>> >>> Avocado-devel at redhat.com <mailto:Avocado-devel at redhat.com>
>>>> >>> https://www.redhat.com/mailman/listinfo/avocado-devel
>>>> >>
>>>> >> _______________________________________________
>>>> >> Avocado-devel mailing list
>>>> >> Avocado-devel at redhat.com <mailto:Avocado-devel at redhat.com>
>>>> >> https://www.redhat.com/mailman/listinfo/avocado-devel
>>>> >
>>>> > _______________________________________________
>>>> > Avocado-devel mailing list
>>>> > Avocado-devel at redhat.com <mailto:Avocado-devel at redhat.com>
>>>> > https://www.redhat.com/mailman/listinfo/avocado-devel
>>>>
>>>> _______________________________________________
>>>> Avocado-devel mailing list
>>>> Avocado-devel at redhat.com <mailto:Avocado-devel at redhat.com>
>>>> https://www.redhat.com/mailman/listinfo/avocado-devel
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Avocado-devel mailing list
>>>> Avocado-devel at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/avocado-devel
>>>>
>>>
>>> _______________________________________________
>>> Avocado-devel mailing list
>>> Avocado-devel at redhat.com
>>> https://www.redhat.com/mailman/listinfo/avocado-devel
>>
>> _______________________________________________
>> Avocado-devel mailing list
>> Avocado-devel at redhat.com
>> https://www.redhat.com/mailman/listinfo/avocado-devel
>
> _______________________________________________
> Avocado-devel mailing list
> Avocado-devel at redhat.com
> https://www.redhat.com/mailman/listinfo/avocado-devel
More information about the Avocado-devel
mailing list