[Pulp-list] Installing Pulp 3.12.0 with ansible and a proxy server

Wed Apr 28 08:22:01 UTC 2021

Hi Ben,

Thanks so much for the update.

Is there any more information you could give us so we can update our docs?
It would be a great help to those who might try this in the future.

All the best,
Melanie

Ar Céad 28 Aib 2021 ag 09:18, scríobh Ben Stanley <ben.stanley at gmail.com>:

> I have now worked around these issues, mostly by manually performing steps
> on the command line or hacking the ansible scripts as previously described.
> I have now managed to install pulp3. It wasn't easy.
>
> Ben.
> On 20/4/21 5:55 am, Mike DePaulo wrote:
>
> Hi Ben,
>
> I have experience dealing with http & https proxies in the past. I would
> very much like to make pulp_installer work properly with them, or to
> provide instructions on how to use them with it.
>
> It seems like when software is configured internally to use a proxy, it
> works. But when software is relying on environment variables, the ansible
> become (i.e., sudo from "user1", to "root", to "pulp") gets rid of the
> environment variable.
>
> Try setting http_proxy and https_proxy as part of the user's environment
> on the system, and configuring sudoers per this comment:
> https://github.com/ansible/ansible/issues/38050#issuecomment-768501547
>
> See in-line replies.
>
> On Sun, Apr 18, 2021 at 10:14 PM Ben Stanley <ben.stanley at gmail.com>
> wrote:
>
>> Hello Pulp people,
>>
>> I'm trying to install pulp 3.12.0 on RHEL 7.8 using the ansible method
>> documented at
>> https://docs.pulpproject.org/pulpcore/installation/instructions.html .
>>
>> I have not yet managed to make it to the end of the pulp_install.yml
>> playbook without error. I have worked around 2 errors, but now I am stuck
>> on the third. I believe the root cause of my problems is trying to use a
>> proxy server. I have set the environment variables http_proxy, https_proxy
>> and proxy appropriately.
>>
>>    1. At the step "TASK [pulp.pulp_installer.pulp_common : Import
>>    required EPEL RPM GPG keys]"
>>    (~/.ansible.collections/ansible_collections/pulp/pulp_installer/roles/pulp_common/tasks/repos.yml),
>>    the rpm_key module has two problems.
>>       1. The ansible rpm_key module fails to pass the proxy settings to
>>       the underlying rpm call.
>>       https://github.com/ansible/ansible/issules/19000
>>       I worked around this problem by replacing the rpm_key ansible
>>       module call with a raw line calling the rpm command directly, and
>>       specifying the proxy settings to use.
>>
>> See the link above for the environment variables.
>
>>
>>    1.
>>       2. The rpm --import <key-url> command uses curl internally.
>>       curl+proxy+https does not work, but curl+proxy+http works. Note
>>       also wget+proxy+https works.
>>
>>       https://unix.stackexchange.com/questions/441021/curling-a-https-url-via-a-proxy-results-in-nss-error-5938
>>       I worked around this problem by referencing the RPM-GPG key with a
>>       http URL instead of a https URL.
>>
>> That sounds like a bug in curl or libcurl. But if you are using a proxy
> for https, then your system is talking to the proxy, which is in turn
> talking to the webserver. So SSL is from your system to the proxy. I
> suspect it's a cipher mismatch per that bug. Let me know if you can figure
> out how to force the cipher.
>
> Either way, I will discuss changing the URL from https to http, or making
> it configurable via a variable at our next installer development meeting.
>
>
>>    1. At the step "TASK [pulp.pulp_installer.pulp_common : Upgrade to a
>>    recent edition of pip (supporting manylinux2014)]"
>>    (~/.ansible.collections/ansible_collections/pulp/pulp_installer/roles/pulp_common/tasks/install_pip.yml),
>>    ansible fails with the error text:
>>    fatal: [honeybee]: FAILED! => {"changed": false, "cmd":
>>    ["/usr/local/lib/pulp/bin/pip", "install", "pip>20.2"], "msg": "stdout:
>>    Collecting pip>20.2\n\n:stderr:   Retrying (Retry(total=4, connect=None,
>>    read=None, redirect=None, status=None)) after connection broken by
>>    'ConnectTimeoutError(<pip._vendor.urllib3.connection.VerifiedHTTPSConnection
>>    object at 0x7ffafd356dd8>, 'Connection to pypi.python.org timed out.
>>    (connect timeout=15)')': /simple/pip/\n  Retrying (Retry(total=3,
>>    connect=None, read=None, redirect=None, status=None)) after connection
>>    broken by
>>    'NewConectionError('<pip.vendor.urllib3.connection.VerifiedHTTPSConnection
>>    object at 0x7ffafd356ef0>: Failed to establish a new connection: [Errno
>>    101] Network is unreachable',)': /simple/pip/\n  Retrying (Retry(total=1,
>>    connect=None, read=None, redirect=None, status=None)) after connection
>>    broken by
>>    'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection
>>    object a 0x7ffafd356f98>:Failed to establish a new connection: [Errno 101]
>>    Network is unreachable .....
>>    I have not figured out how to work around this problem. It seems that
>>    the pip ansible command is also not passing on the correct proxy settings.
>>    I haven't even figured out how to work around this problem running pip
>>    manually yet.
>>
>> Hmm, so we start out with the old system version of pip, copied into the
> virtualenv. Then we use it to upgrade the virtualenv the new version of pip.
>
> Perhaps the old version cannot talk to the proxy?
>
> Try using the virtualenv like:
> sudo -i -u pulp
> source /usr/local/lib/pulp/bin/activate
> export http_proxy=your-proxy-url
> export https_proxy=your-proxy-url
> pip install --upgrade pip
>
>> It would be fantastic if I could get some help with these issues so that
>> I can get my pulp server upgraded from pulp2 to pulp3.
>>
>> Thanks,
>> Ben Stanley.
>>
>
> -Mike
>
> --
>
> Mike DePaulo
>
> He / Him / His
>
> Service Reliability Engineer, Pulp
>
> Red Hat <https://www.redhat.com/>
>
> IM: mikedep333
>
> GPG: 51745404
> <https://www.redhat.com/>
>
> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-list

-- 

Melanie Corr, RHCE

Community Manager

Red Hat <https://www.redhat.com>

Remote, Ireland

mcorr at redhat.com
M: +353857774436     IM: mcorr
<https://www.redhat.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20210428/8998458f/attachment.htm>