[Pulp-list] Installing Pulp 3.12.0 with ansible and a proxy server

[Mike DePaulo]

Hi Ben,

I have experience dealing with http & https proxies in the past. I would
very much like to make pulp_installer work properly with them, or to
provide instructions on how to use them with it.

It seems like when software is configured internally to use a proxy, it
works. But when software is relying on environment variables, the ansible
become (i.e., sudo from "user1", to "root", to "pulp") gets rid of the
environment variable.

Try setting http_proxy and https_proxy as part of the user's environment on
the system, and configuring sudoers per this comment:
https://github.com/ansible/ansible/issues/38050#issuecomment-768501547

See in-line replies.

On Sun, Apr 18, 2021 at 10:14 PM Ben Stanley <ben.stanley at gmail.com> wrote:

> Hello Pulp people,
>
> I'm trying to install pulp 3.12.0 on RHEL 7.8 using the ansible method
> documented at
> https://docs.pulpproject.org/pulpcore/installation/instructions.html .
>
> I have not yet managed to make it to the end of the pulp_install.yml
> playbook without error. I have worked around 2 errors, but now I am stuck
> on the third. I believe the root cause of my problems is trying to use a
> proxy server. I have set the environment variables http_proxy, https_proxy
> and proxy appropriately.
>
>    1. At the step "TASK [pulp.pulp_installer.pulp_common : Import
>    required EPEL RPM GPG keys]"
>    (~/.ansible.collections/ansible_collections/pulp/pulp_installer/roles/pulp_common/tasks/repos.yml),
>    the rpm_key module has two problems.
>       1. The ansible rpm_key module fails to pass the proxy settings to
>       the underlying rpm call.
>       https://github.com/ansible/ansible/issules/19000
>       I worked around this problem by replacing the rpm_key ansible
>       module call with a raw line calling the rpm command directly, and
>       specifying the proxy settings to use.
>
> See the link above for the environment variables.

>
>    1.
>       2. The rpm --import <key-url> command uses curl internally.
>       curl+proxy+https does not work, but curl+proxy+http works. Note
>       also wget+proxy+https works.
>
>       https://unix.stackexchange.com/questions/441021/curling-a-https-url-via-a-proxy-results-in-nss-error-5938
>       I worked around this problem by referencing the RPM-GPG key with a
>       http URL instead of a https URL.
>
> That sounds like a bug in curl or libcurl. But if you are using a proxy
for https, then your system is talking to the proxy, which is in turn
talking to the webserver. So SSL is from your system to the proxy. I
suspect it's a cipher mismatch per that bug. Let me know if you can figure
out how to force the cipher.

Either way, I will discuss changing the URL from https to http, or making
it configurable via a variable at our next installer development meeting.


>    1. At the step "TASK [pulp.pulp_installer.pulp_common : Upgrade to a
>    recent edition of pip (supporting manylinux2014)]"
>    (~/.ansible.collections/ansible_collections/pulp/pulp_installer/roles/pulp_common/tasks/install_pip.yml),
>    ansible fails with the error text:
>    fatal: [honeybee]: FAILED! => {"changed": false, "cmd":
>    ["/usr/local/lib/pulp/bin/pip", "install", "pip>20.2"], "msg": "stdout:
>    Collecting pip>20.2\n\n:stderr:   Retrying (Retry(total=4, connect=None,
>    read=None, redirect=None, status=None)) after connection broken by
>    'ConnectTimeoutError(<pip._vendor.urllib3.connection.VerifiedHTTPSConnection
>    object at 0x7ffafd356dd8>, 'Connection to pypi.python.org timed out.
>    (connect timeout=15)')': /simple/pip/\n  Retrying (Retry(total=3,
>    connect=None, read=None, redirect=None, status=None)) after connection
>    broken by
>    'NewConectionError('<pip.vendor.urllib3.connection.VerifiedHTTPSConnection
>    object at 0x7ffafd356ef0>: Failed to establish a new connection: [Errno
>    101] Network is unreachable',)': /simple/pip/\n  Retrying (Retry(total=1,
>    connect=None, read=None, redirect=None, status=None)) after connection
>    broken by
>    'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection
>    object a 0x7ffafd356f98>:Failed to establish a new connection: [Errno 101]
>    Network is unreachable .....
>    I have not figured out how to work around this problem. It seems that
>    the pip ansible command is also not passing on the correct proxy settings.
>    I haven't even figured out how to work around this problem running pip
>    manually yet.
>
> Hmm, so we start out with the old system version of pip, copied into the
virtualenv. Then we use it to upgrade the virtualenv the new version of pip.

Perhaps the old version cannot talk to the proxy?

Try using the virtualenv like:
sudo -i -u pulp
source /usr/local/lib/pulp/bin/activate
export http_proxy=your-proxy-url
export https_proxy=your-proxy-url
pip install --upgrade pip

> It would be fantastic if I could get some help with these issues so that I
> can get my pulp server upgraded from pulp2 to pulp3.
>
> Thanks,
> Ben Stanley.
>

-Mike

-- 

Mike DePaulo

He / Him / His

Service Reliability Engineer, Pulp

Red Hat <https://www.redhat.com/>

IM: mikedep333

GPG: 51745404
<https://www.redhat.com/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20210419/9093ed61/attachment.htm>