[Pulp-list] pulp 3.2 to 3.3.1 pip-compile error
Bin Li (BLOOMBERG/ 120 PARK)
bli111 at bloomberg.net
Fri May 29 21:33:39 UTC 2020
Thanks Mike, This works. On a another host, I had to remove the current installation dir and install from scratch and it works. The new installation also has pip-tools 5.2. Is it possible to have prerequisite in pulp_installer so it is always upgraded first?
From: mikedep333 at redhat.com At: 05/29/20 17:24:29To: Bin Li (BLOOMBERG/ 120 PARK )
Cc: pulp-list at redhat.com
Subject: Re: [Pulp-list] pulp 3.2 to 3.3.1 pip-compile error
Hmm, your pip-tools is much older:
pip-tools 5.2.0
It looks like we need to modify the task that installs pip-tools to ensure it is new enough. The changelog says that pip-tools 5.0 is about pip 20.0 support.
In the meantime, an easy workaround is:
$ source /usr/local/lib/pulp/bin/activate
$ pip install --upgrade pip-tools
Then re-run the installer.
-Mike
On Fri, May 29, 2020 at 5:17 PM Bin Li (BLOOMBERG/ 120 PARK) <bli111 at bloomberg.net> wrote:
Thanks Mike. Here is the output.
# more /opt/utils/venv/osupgrade/3.7.3/pyvenv.cfg
home = /opt/python/3.7.3/bin
include-system-site-packages = false
version = 3.7.3
Package Version
------------------------------- ------------
aiodns 2.0.0
aiofiles 0.4.0
aiohttp 3.6.2
ansible 2.9.6
async-timeout 3.0.1
attrs 19.3.0
backoff 1.10.0
certifi 2019.11.28
cffi 1.14.0
chardet 3.0.4
Click 7.0
coreapi 2.3.3
coreschema 0.0.4
createrepo-c 0.15.7
cryptography 2.8
Django 2.2.11
django-filter 2.2.0
djangorestframework 3.10.3
djangorestframework-queryfields 1.0.0
drf-nested-routers 0.91
drf-yasg 1.17.1
dynaconf 3.0.0rc1
gunicorn 20.0.4
idna 2.9
importlib-metadata 1.5.0
inflection 0.3.1
itypes 1.1.0
Jinja2 2.11.1
jsonschema 3.2.0
libcomps 0.1.14.post1
MarkupSafe 1.1.1
modulemd 1.3.3
multidict 4.7.5
nose 1.3.7
packaging 20.3
pip 20.1.1
pip-tools 4.5.1
productmd 1.24
psycopg2 2.8.4
pulp-file 0.2.0
pulp-file-client 0.2.0
pulp-rpm 3.2.0
pulp-rpm-client 3.2.0
pulpcore 3.2.0
pulpcore-client 3.2.0
pycairo 1.19.1
pycares 3.1.1
pycparser 2.20
PyGObject 3.34.0
pygtrie 2.3.2
pyparsing 2.4.6
pyrsistent 0.15.7
python-box 3.4.6
python-dateutil 2.8.1
python-dotenv 0.12.0
python-gnupg 0.4.5
pytz 2019.3
PyYAML 5.3
redis 3.4.1
requests 2.23.0
rq 1.2.2
ruamel.yaml 0.16.10
ruamel.yaml.clib 0.2.0
scikit-build 0.10.0
setuptools 40.8.0
six 1.14.0
sqlparse 0.3.1
toml 0.10.0
uritemplate 3.0.1
urllib3 1.25.8
wheel 0.34.2
whitenoise 5.0.1
yarl 1.4.2
zipp 3.1.0
From: mikedep333 at redhat.com At: 05/29/20 17:08:46To: Bin Li (BLOOMBERG/ 120 PARK )
Cc: pulp-list at redhat.com
Subject: Re: [Pulp-list] pulp 3.2 to 3.3.1 pip-compile error
1. I ran a test install against CentOS 7.6. Now we do not support CentOS 7.6 and earlier anymore, 7.7 & 7.8 work. RHEL is in the same situation. I disabled the check CentOS version task, and newer packages from the CentOS 7.8 repos were installed (like those for python3 and for libmodulem2), so it actually succeeded. I did not get the error like we would expect on the old distro (due to the newer packages on top), and I did not get your error either:
https://pastebin.com/uBtQ9UTv
2. My apologies, I mistyped it. It is pyvenv.cfg , with a v:
[vagrant at pulp3-sandbox-centos7 ~]$ cat /usr/local/lib/pulp/pyvenv.cfg
home = /usr/bin
include-system-site-packages = false
version = 3.6.8
Please run that.
3. I'm going to have to think about any other steps we can take to reproduce your problem. Can you run the following?
$ source /usr/local/lib/pulp/bin/activate
(pulp) $ pip list
On Fri, May 29, 2020 at 4:17 PM Bin Li (BLOOMBERG/ 120 PARK) <bli111 at bloomberg.net> wrote:
Mike,1. The pulp server is on rhel7.6
2. pulp_use_system_wide_pkgs is not defined. I run a find to grep pulp_use_system_wide_pkgs and didn't see it defined in ansible roles
I don't find this file /usr/local/lib/pulp/pyenv.cfg either. I also run a find and didn't have a file named pyenv.cfg.
From: mikedep333 at redhat.com At: 05/29/20 15:09:42To: Bin Li (BLOOMBERG/ 120 PARK )
Cc: pulp-list at redhat.com
Subject: Re: [Pulp-list] pulp 3.2 to 3.3.1 pip-compile error
Your pulp_instal_plugins looks correct for Pulp 3.3.1.
Interesting error, I will try to reproduce.
2 questions:
1. What distro are you installing Pulp on?
2. pulp_use_system_wide_pkgs is undefined or set to false, correct? If true, it causes all sorts of problems with our pip-compile check, is no longer needed for pulp-rpm to install, and we are about to remove it.
If you previously set it to true, and are upgrading a system, it would still be effectively true as well. In fact, until 3.3.0, we were setting it to true in the pulp_rpm_prerequisites role, even if you didn't set it. So that would be another reason for it to be effectively true. You can check by viewing the file /usr/local/lib/pulp/pyenv.cfg .
-Mike
On Fri, May 29, 2020 at 11:52 AM Bin Li (BLOOMBERG/ 120 PARK) <bli111 at bloomberg.net> wrote:
I have both pulp-file and pulp-rpm pined to compatible version.
pulp_install_plugins:
pulp-file:
version: '0.3.0'
pulp-rpm:
version: '3.3.2'
Still got an error as below. Please advise
TASK [pulp : Run pip-compile to check pulpcore/plugin compatibility] ****************************************************************
[WARNING]: conditional statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ failed_condition
| default("compatibility.rc != 0") }}
fatal: [myhost]: FAILED! => {"changed": false, "cmd": ["/opt/utils/venv/pulp/3.7.3/bin/pip-compile"], "delta": "0:00:00.393598", "end": "2020-05-29 11:46:09.917436", "failed_when_result": true, "msg": "non-zero return code", "rc": 1, "start": "2020-05-29 11:46:09.523838", "stderr": "Traceback (most recent call last):\n File \"/opt/utils/venv/pulp/3.7.3/bin/pip-compile\", line 8, in <module>\n sys.exit(cli())\n File \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/click/core.py\", line 764, in __call__\n return self.main(*args, **kwargs)\n File \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/click/core.py\", line 717, in main\n rv = self.invoke(ctx)\n File \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/click/core.py\", line 956, in invoke\n return ctx.invoke(self.callback, **ctx.params)\n File \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/click/core.py\", line 555, in invoke\n return callback(*args, **kwargs)\n File \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/click/decorators.py\", line 17, in new_func\n return f(get_current_context(), *args, **kwargs)\n File \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/piptools/scripts/compile.py\", line 304, in cli\n for ireq in filter(is_pinned_requirement, ireqs):\n File \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/piptools/utils.py\", line 122, in is_pinned_requirement\n if ireq.editable:\nAttributeError: 'ParsedRequirement' object has no attribute 'editable'", "stderr_lines": ["Traceback (most recent call last):", " File \"/opt/utils/venv/pulp/3.7.3/bin/pip-compile\", line 8, in <module>", " sys.exit(cli())", " File \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/click/core.py\", line 764, in __call__", " return self.main(*args, **kwargs)", " File \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/click/core.py\", line 717, in main", " rv = self.invoke(ctx)", " File \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/click/core.py\", line 956, in invoke", " return ctx.invoke(self.callback, **ctx.params)", " File \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/click/core.py\", line 555, in invoke", " return callback(*args, **kwargs)", " File \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/click/decorators.py\", line 17, in new_func", " return f(get_current_context(), *args, **kwargs)", " File \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/piptools/scripts/compile.py\", line 304, in cli", " for ireq in filter(is_pinned_requirement, ireqs):", " File \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/piptools/utils.py\", line 122, in is_pinned_requirement", " if ireq.editable:", "AttributeError: 'ParsedRequirement' object has no attribute 'editable'"], "stdout": "", "stdout_lines": []}
_______________________________________________
Pulp-list mailing list
Pulp-list at redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list
--
Mike DePaulo
He / Him / His
Service Reliability Engineer, Pulp
Red Hat
IM: mikedep333
GPG: 51745404
--
Mike DePaulo
He / Him / His
Service Reliability Engineer, Pulp
Red Hat
IM: mikedep333
GPG: 51745404
--
Mike DePaulo
He / Him / His
Service Reliability Engineer, Pulp
Red Hat
IM: mikedep333
GPG: 51745404
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20200529/696d03f7/attachment.htm>
More information about the Pulp-list
mailing list