[Pulp-list] pulp 3.2 to 3.3.1 pip-compile error
Mike DePaulo
mikedep333 at redhat.com
Mon Jun 1 18:57:02 UTC 2020
FYI, this has been merged into the master branch of pulp_installer. It will
be in the next release:
https://pulp.plan.io/issues/6864
On Fri, May 29, 2020 at 5:44 PM Mike DePaulo <mikedep333 at redhat.com> wrote:
> Absolutely, I'll just modify the current task that installs pip-tools.
> Later on, pip-tools is used to perform the check.
>
> https://github.com/pulp/pulp_installer/blob/master/roles/pulp/tasks/install.yml#L182
>
> -Mike
>
> On Fri, May 29, 2020 at 5:33 PM Bin Li (BLOOMBERG/ 120 PARK) <
> bli111 at bloomberg.net> wrote:
>
>> Thanks Mike, This works. On a another host, I had to remove the current
>> installation dir and install from scratch and it works. The new
>> installation also has pip-tools 5.2. Is it possible to have prerequisite in
>> pulp_installer so it is always upgraded first?
>>
>> From: mikedep333 at redhat.com At: 05/29/20 17:24:29
>> To: Bin Li (BLOOMBERG/ 120 PARK ) <bli111 at bloomberg.net>
>> Cc: pulp-list at redhat.com
>> Subject: Re: [Pulp-list] pulp 3.2 to 3.3.1 pip-compile error
>>
>> Hmm, your pip-tools is much older:
>> pip-tools 5.2.0
>>
>> It looks like we need to modify the task that installs pip-tools to
>> ensure it is new enough. The changelog says that pip-tools 5.0 is about pip
>> 20.0 support.
>>
>> In the meantime, an easy workaround is:
>> $ source /usr/local/lib/pulp/bin/activate
>> $ pip install --upgrade pip-tools
>>
>> Then re-run the installer.
>>
>> -Mike
>>
>> On Fri, May 29, 2020 at 5:17 PM Bin Li (BLOOMBERG/ 120 PARK) <
>> bli111 at bloomberg.net> wrote:
>>
>>> Thanks Mike. Here is the output.
>>>
>>> # more /opt/utils/venv/osupgrade/3.7.3/pyvenv.cfg
>>> home = /opt/python/3.7.3/bin
>>> include-system-site-packages = false
>>> version = 3.7.3
>>>
>>> Package Version
>>> ------------------------------- ------------
>>> aiodns 2.0.0
>>> aiofiles 0.4.0
>>> aiohttp 3.6.2
>>> ansible 2.9.6
>>> async-timeout 3.0.1
>>> attrs 19.3.0
>>> backoff 1.10.0
>>> certifi 2019.11.28
>>> cffi 1.14.0
>>> chardet 3.0.4
>>> Click 7.0
>>> coreapi 2.3.3
>>> coreschema 0.0.4
>>> createrepo-c 0.15.7
>>> cryptography 2.8
>>> Django 2.2.11
>>> django-filter 2.2.0
>>> djangorestframework 3.10.3
>>> djangorestframework-queryfields 1.0.0
>>> drf-nested-routers 0.91
>>> drf-yasg 1.17.1
>>> dynaconf 3.0.0rc1
>>> gunicorn 20.0.4
>>> idna 2.9
>>> importlib-metadata 1.5.0
>>> inflection 0.3.1
>>> itypes 1.1.0
>>> Jinja2 2.11.1
>>> jsonschema 3.2.0
>>> libcomps 0.1.14.post1
>>> MarkupSafe 1.1.1
>>> modulemd 1.3.3
>>> multidict 4.7.5
>>> nose 1.3.7
>>> packaging 20.3
>>> pip 20.1.1
>>> pip-tools 4.5.1
>>> productmd 1.24
>>> psycopg2 2.8.4
>>> pulp-file 0.2.0
>>> pulp-file-client 0.2.0
>>> pulp-rpm 3.2.0
>>> pulp-rpm-client 3.2.0
>>> pulpcore 3.2.0
>>> pulpcore-client 3.2.0
>>> pycairo 1.19.1
>>> pycares 3.1.1
>>> pycparser 2.20
>>> PyGObject 3.34.0
>>> pygtrie 2.3.2
>>> pyparsing 2.4.6
>>> pyrsistent 0.15.7
>>> python-box 3.4.6
>>> python-dateutil 2.8.1
>>> python-dotenv 0.12.0
>>> python-gnupg 0.4.5
>>> pytz 2019.3
>>> PyYAML 5.3
>>> redis 3.4.1
>>> requests 2.23.0
>>> rq 1.2.2
>>> ruamel.yaml 0.16.10
>>> ruamel.yaml.clib 0.2.0
>>> scikit-build 0.10.0
>>> setuptools 40.8.0
>>> six 1.14.0
>>> sqlparse 0.3.1
>>> toml 0.10.0
>>> uritemplate 3.0.1
>>> urllib3 1.25.8
>>> wheel 0.34.2
>>> whitenoise 5.0.1
>>> yarl 1.4.2
>>> zipp 3.1.0
>>>
>>>
>>>
>>> From: mikedep333 at redhat.com At: 05/29/20 17:08:46
>>> To: Bin Li (BLOOMBERG/ 120 PARK ) <bli111 at bloomberg.net>
>>> Cc: pulp-list at redhat.com
>>> Subject: Re: [Pulp-list] pulp 3.2 to 3.3.1 pip-compile error
>>>
>>>
>>> 1. I ran a test install against CentOS 7.6. Now we do not support
>>> CentOS 7.6 and earlier anymore, 7.7 & 7.8 work. RHEL is in the same
>>> situation. I disabled the check CentOS version task, and newer packages
>>> from the CentOS 7.8 repos were installed (like those for python3 and for
>>> libmodulem2), so it actually succeeded. I did not get the error like we
>>> would expect on the old distro (due to the newer packages on top), and I
>>> did not get your error either:
>>> https://pastebin.com/uBtQ9UTv
>>>
>>> 2. My apologies, I mistyped it. It is pyvenv.cfg , with a v:
>>> [vagrant at pulp3-sandbox-centos7 ~]$ cat /usr/local/lib/pulp/pyvenv.cfg
>>> home = /usr/bin
>>> include-system-site-packages = false
>>> version = 3.6.8
>>>
>>> Please run that.
>>>
>>> 3. I'm going to have to think about any other steps we can take to
>>> reproduce your problem. Can you run the following?
>>> $ source /usr/local/lib/pulp/bin/activate
>>> (pulp) $ pip list
>>>
>>> On Fri, May 29, 2020 at 4:17 PM Bin Li (BLOOMBERG/ 120 PARK) <
>>> bli111 at bloomberg.net> wrote:
>>>
>>>> Mike,
>>>> 1. The pulp server is on rhel7.6
>>>> 2. pulp_use_system_wide_pkgs is not defined. I run a find to grep
>>>> pulp_use_system_wide_pkgs and didn't see it defined in ansible roles
>>>> I don't find this file /usr/local/lib/pulp/pyenv.cfg either. I also run
>>>> a find and didn't have a file named pyenv.cfg.
>>>>
>>>>
>>>> From: mikedep333 at redhat.com At: 05/29/20 15:09:42
>>>> To: Bin Li (BLOOMBERG/ 120 PARK ) <bli111 at bloomberg.net>
>>>> Cc: pulp-list at redhat.com
>>>> Subject: Re: [Pulp-list] pulp 3.2 to 3.3.1 pip-compile error
>>>>
>>>> Your pulp_instal_plugins looks correct for Pulp 3.3.1.
>>>>
>>>> Interesting error, I will try to reproduce.
>>>>
>>>> 2 questions:
>>>> 1. What distro are you installing Pulp on?
>>>> 2. pulp_use_system_wide_pkgs is undefined or set to false, correct? If
>>>> true, it causes all sorts of problems with our pip-compile check, is no
>>>> longer needed for pulp-rpm to install, and we are about to remove it.
>>>> If you previously set it to true, and are upgrading a system, it would
>>>> still be effectively true as well. In fact, until 3.3.0, we were setting it
>>>> to true in the pulp_rpm_prerequisites role, even if you didn't set it. So
>>>> that would be another reason for it to be effectively true. You can check
>>>> by viewing the file /usr/local/lib/pulp/pyenv.cfg .
>>>>
>>>> -Mike
>>>>
>>>> On Fri, May 29, 2020 at 11:52 AM Bin Li (BLOOMBERG/ 120 PARK) <
>>>> bli111 at bloomberg.net> wrote:
>>>>
>>>>> I have both pulp-file and pulp-rpm pined to compatible version.
>>>>> pulp_install_plugins:
>>>>> pulp-file:
>>>>> version: '0.3.0'
>>>>> pulp-rpm:
>>>>> version: '3.3.2'
>>>>>
>>>>> Still got an error as below. Please advise
>>>>>
>>>>> TASK [pulp : Run pip-compile to check pulpcore/plugin compatibility]
>>>>> ****************************************************************
>>>>> [WARNING]: conditional statements should not include jinja2 templating
>>>>> delimiters such as {{ }} or {% %}. Found: {{ failed_condition
>>>>> | default("compatibility.rc != 0") }}
>>>>> fatal: [myhost]: FAILED! => {"changed": false, "cmd":
>>>>> ["/opt/utils/venv/pulp/3.7.3/bin/pip-compile"], "delta": "0:00:00.393598",
>>>>> "end": "2020-05-29 11:46:09.917436", "failed_when_result": true, "msg":
>>>>> "non-zero return code", "rc": 1, "start": "2020-05-29 11:46:09.523838",
>>>>> "stderr": "Traceback (most recent call last):\n File
>>>>> \"/opt/utils/venv/pulp/3.7.3/bin/pip-compile\", line 8, in <module>\n
>>>>> sys.exit(cli())\n File
>>>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/click/core.py\",
>>>>> line 764, in __call__\n return self.main(*args, **kwargs)\n File
>>>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/click/core.py\",
>>>>> line 717, in main\n rv = self.invoke(ctx)\n File
>>>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/click/core.py\",
>>>>> line 956, in invoke\n return ctx.invoke(self.callback, **ctx.params)\n File
>>>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/click/core.py\",
>>>>> line 555, in invoke\n return callback(*args, **kwargs)\n File
>>>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/click/decorators.py\",
>>>>> line 17, in new_func\n return f(get_current_context(), *args, **kwargs)\n
>>>>> File
>>>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/piptools/scripts/compile.py\",
>>>>> line 304, in cli\n for ireq in filter(is_pinned_requirement, ireqs):\n File
>>>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/piptools/utils.py\",
>>>>> line 122, in is_pinned_requirement\n if ireq.editable:\nAttributeError:
>>>>> 'ParsedRequirement' object has no attribute 'editable'", "stderr_lines":
>>>>> ["Traceback (most recent call last):", " File
>>>>> \"/opt/utils/venv/pulp/3.7.3/bin/pip-compile\", line 8, in <module>", "
>>>>> sys.exit(cli())", " File
>>>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/click/core.py\",
>>>>> line 764, in __call__", " return self.main(*args, **kwargs)", " File
>>>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/click/core.py\",
>>>>> line 717, in main", " rv = self.invoke(ctx)", " File
>>>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/click/core.py\",
>>>>> line 956, in invoke", " return ctx.invoke(self.callback, **ctx.params)", "
>>>>> File
>>>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/click/core.py\",
>>>>> line 555, in invoke", " return callback(*args, **kwargs)", " File
>>>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/click/decorators.py\",
>>>>> line 17, in new_func", " return f(get_current_context(), *args, **kwargs)",
>>>>> " File
>>>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/piptools/scripts/compile.py\",
>>>>> line 304, in cli", " for ireq in filter(is_pinned_requirement, ireqs):", "
>>>>> File
>>>>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/piptools/utils.py\",
>>>>> line 122, in is_pinned_requirement", " if ireq.editable:", "AttributeError:
>>>>> 'ParsedRequirement' object has no attribute 'editable'"], "stdout": "",
>>>>> "stdout_lines": []}
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Pulp-list mailing list
>>>>> Pulp-list at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/pulp-list
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Mike DePaulo
>>>>
>>>> He / Him / His
>>>>
>>>> Service Reliability Engineer, Pulp
>>>>
>>>> Red Hat <https://www.redhat.com/>
>>>>
>>>> IM: mikedep333
>>>>
>>>> GPG: 51745404
>>>> <https://www.redhat.com/>
>>>>
>>>>
>>>>
>>>
>>> --
>>>
>>> Mike DePaulo
>>>
>>> He / Him / His
>>>
>>> Service Reliability Engineer, Pulp
>>>
>>> Red Hat <https://www.redhat.com/>
>>>
>>> IM: mikedep333
>>>
>>> GPG: 51745404
>>> <https://www.redhat.com/>
>>>
>>>
>>>
>>
>> --
>>
>> Mike DePaulo
>>
>> He / Him / His
>>
>> Service Reliability Engineer, Pulp
>>
>> Red Hat <https://www.redhat.com/>
>>
>> IM: mikedep333
>>
>> GPG: 51745404
>> <https://www.redhat.com/>
>>
>>
>>
>
> --
>
> Mike DePaulo
>
> He / Him / His
>
> Service Reliability Engineer, Pulp
>
> Red Hat <https://www.redhat.com/>
>
> IM: mikedep333
>
> GPG: 51745404
> <https://www.redhat.com/>
>
--
Mike DePaulo
He / Him / His
Service Reliability Engineer, Pulp
Red Hat <https://www.redhat.com/>
IM: mikedep333
GPG: 51745404
<https://www.redhat.com/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20200601/830a5381/attachment.htm>
More information about the Pulp-list
mailing list