[Pulp-list] create remote to redhat on pulp 3

Sun Jun 23 11:14:37 UTC 2019

The problem is that the proxy setting is not being respected by Pulp. I
filed an issue about this[0] and also made a patch to fix it[1]. This patch
should be merged in the next 24 hours.

If you cannot access cdn.redhat.com without a proxy, the error you got
makes sense. I was only able to reproduce the error message when I added an
incorrect entry for cdn.redhat.com in /etc/hosts.

[0] https://pulp.plan.io/issues/5011
[1] https://github.com/pulp/pulpcore-plugin/pull/107

On Fri, Jun 21, 2019 at 12:26 PM Bin Li (BLOOMBERG/ 120 PARK) <
bli111 at bloomberg.net> wrote:

> Thanks. I am now getting a different error. Please advise
>
> "_href": "/pulp/api/v3/tasks/fcc679b8-1ad3-4a16-834b-47b946caaeed/",
> "created_resources": [],
> "error": {
> "code": null,
> "description": "Cannot connect to host cdn.redhat.com:443 ssl:None
> [Connect call failed ('23.65.16.251', 443)]",
> "traceback": " File
> \"/opt/python/3.6.5/lib/python3.6/site-packages/rq/worker.py\", line 812,
> in perform_job\n rv = job.perform()\n File
> \"/opt/python/3.6.5/lib/python3.6/site-packages/rq/job.py\", line 588, in
> perform\n self._result = self._execute()\n File
> \"/opt/python/3.6.5/lib/python3.6/site-packages/rq/job.py\", line 594, in
> _execute\n return self.func(*self.args, **self.kwargs)\n File
> \"/tmp/pulp_rpm/pulp_rpm/app/tasks/synchronizing.py\", line 67, in
> synchronize\n dv.create()\n File
> \"/tmp/pulpcore-plugin/pulpcore/plugin/stages/declarative_version.py\",
> line 169, in create\n loop.run_until_complete(pipeline)\n File
> \"/opt/python/3.6.5/lib64/python3.6/asyncio/base_events.py\", line 468, in
> run_until_complete\n return future.result()\n File
> \"/tmp/pulpcore-plugin/pulpcore/plugin/stages/api.py\", line 209, in
> create_pipeline\n await asyncio.gather(*futures)\n File
> \"/tmp/pulpcore-plugin/pulpcore/plugin/stages/api.py\", line 43, in
> __call__\n await self.run()\n File
> \"/tmp/pulp_rpm/pulp_rpm/app/tasks/synchronizing.py\", line 231, in run\n
> result = await downloader.run()\n File
> \"/tmp/pulpcore-plugin/pulpcore/plugin/download/base.py\", line 212, in
> run\n return await self._run(extra_data=extra_data)\n File
> \"/opt/python/3.6.5/lib/python3.6/site-packages/backoff/_async.py\", line
> 131, in retry\n ret = await target(*args, **kwargs)\n File
> \"/tmp/pulpcore-plugin/pulpcore/plugin/download/http.py\", line 182, in
> _run\n async with self.session.get(self.url) as response:\n File
> \"/opt/python/3.6.5/lib64/python3.6/site-packages/aiohttp/client.py\", line
> 1005, in __aenter__\n self._resp = await self._coro\n File
> \"/opt/python/3.6.5/lib64/python3.6/site-packages/aiohttp/client.py\", line
> 476, in _request\n timeout=real_timeout\n File
> \"/opt/python/3.6.5/lib64/python3.6/site-packages/aiohttp/connector.py\",
> line 522, in connect\n proto = await self._create_connection(req, traces,
> timeout)\n File
> \"/opt/python/3.6.5/lib64/python3.6/site-packages/aiohttp/connector.py\",
> line 854, in _create_connection\n req, traces, timeout)\n File
> \"/opt/python/3.6.5/lib64/python3.6/site-packages/aiohttp/connector.py\",
> line 992, in _create_direct_connection\n raise last_exc\n File
> \"/opt/python/3.6.5/lib64/python3.6/site-packages/aiohttp/connector.py\",
> line 974, in _create_direct_connection\n req=req,
> client_error=client_error)\n File
> \"/opt/python/3.6.5/lib64/python3.6/site-packages/aiohttp/connector.py\",
> line 931, in _wrap_create_connection\n raise
> client_error(req.connection_key, exc) from exc\n"
> },
>
>
> I was able to connect using client cert through proxy
> # curl -Iv --cacert /etc/rhsm/ca/redhat-uep.pem --key
> /etc/pki/entitlement/8521692907269500331-key.pem --cert
> /etc/pki/entitlement/8521692907269500331.pem
> https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os/
> * About to connect() to proxy ... port ... (#0)
> * Trying ...
> * Connected to ... () port ... (#0)
> * Establish HTTP proxy tunnel to cdn.redhat.com:443
> > CONNECT cdn.redhat.com:443 HTTP/1.1
> > Host: cdn.redhat.com:443
> > User-Agent: curl/7.29.0
> > Proxy-Connection: Keep-Alive
> >
> < HTTP/1.1 200 Connection established
> HTTP/1.1 200 Connection established
> <
>
> * Proxy replied OK to CONNECT request
> * Initializing NSS with certpath: sql:/etc/pki/nssdb
>
>
> From: dkliban at redhat.com At: 06/20/19 16:34:40
> To: Bin Li (BLOOMBERG/ 120 PARK ) <bli111 at bloomberg.net>
> Cc: pulp-list at redhat.com
> Subject: Re: [Pulp-list] create remote to redhat on pulp 3
>
> You need to make sure that the following services are running:
> pulp-resource-manager, pulp-worker at 0, pulp-worker at 1 (at least 1 worker is
> needed).
>
> On Thu, Jun 20, 2019 at 4:11 PM Bin Li (BLOOMBERG/ 120 PARK) <
> bli111 at bloomberg.net> wrote:
>
>> I installed the latest version of component. No more errors when try to
>> sync the repo but the task is in waiting state forever. Any idea what I
>> should check?
>>
>> "versions": [
>> {
>> "component": "pulpcore",
>> "version": "3.0.0rc3.dev0"
>> },
>> {
>> "component": "pulpcore-plugin",
>> "version": "0.1.0rc3.dev0"
>> },
>> {
>> "component": "pulp_rpm",
>> "version": "3.0.0b4.dev0"
>> }
>>
>>
>>
>> From: dkliban at redhat.com At: 06/18/19 13:42:01
>> To: Bin Li (BLOOMBERG/ 120 PARK ) <bli111 at bloomberg.net>
>> Cc: pulp-list at redhat.com
>> Subject: Re: [Pulp-list] create remote to redhat on pulp 3
>>
>> This issue has now been resolved and a fix is available on master branch
>> of pulpcore-plugin[0].
>>
>>
>> [0] https://github.com/pulp/pulpcore-plugin/tree/master/
>>
>> On Wed, Jun 12, 2019 at 1:51 PM Dennis Kliban <dkliban at redhat.com> wrote:
>>
>>> There is an open issue about this[0]. I recommend clicking the watch
>>> button so you get notifications about it. We should be fixing this soon.
>>>
>>> [0] https://pulp.plan.io/issues/4825
>>>
>>> On Wed, Jun 12, 2019 at 1:42 PM Bin Li (BLOOMBERG/ 120 PARK) <
>>> bli111 at bloomberg.net> wrote:
>>>
>>>>
>>>> I was not able to sync a protected repo from redhat. The host running
>>>> pulp 3 subscribes to redhat directly.
>>>>
>>>> I first tried to use the following to create remote:
>>>> {
>>>> "name": "rhel-x86_64-server-7",
>>>> "url": "
>>>> https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os",
>>>> "ssl_ca_certificate": "/etc/rhsm/ca/redhat-uep.pem",
>>>> "ssl_client_certificate": "/etc/pki/consumer/cert.pem",
>>>> "ssl_client_key": "/etc/pki/consumer/key.pem",
>>>> "ssl_validation": true,
>>>> "proxy_url": "http://proxy:80",
>>>> "download_concurrency": 1,
>>>> "policy": "immediate"
>>>> }
>>>>
>>>> Remote was created but I got errors "Jun 12 12:51:43 ip-1-76-158-244
>>>> rq[19389]: pulp: rq.worker:ERROR: ssl.SSLError: [PEM: NO_START_LINE] no
>>>> start line (_ssl.c:3626)" when I tried to sync with http POST
>>>> :24817${REMOTE_HREF}sync/ repository=$REPO_HREF
>>>>
>>>> Then I tried to converted the pem file to a json format
>>>> {
>>>> "name": "rhel-x86_64-server-7",
>>>> "url": "
>>>> https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os",
>>>> "ssl_ca_certificate": "-----BEGIN
>>>> CERTIFICATE-----\nMIIG/TCCBOWgAwIBAgIBNzANBgkqhkiG9w0BAQUFADCBsTELMAkGA1UEBhMCVVMx\nFzAVBg
>>>> ...
>>>> "ssl_client_certificate": "-----BEGIN
>>>> CERTIFICATE-----\nMIIMojCCCoqgAwIBAgIISqmnKnJ9sEowDQ
>>>> ...
>>>> "ssl_client_key": "-----BEGIN RSA PRIVATE
>>>> KEY-----\nMIIJKQIBAAKCAgEAuujl5HxnIDTSOemZOyH3Jr5xxgrc1rCee
>>>> ...
>>>> "ssl_validation": true,
>>>> "proxy_url": "http://proxy:80",
>>>> "download_concurrency": 1,
>>>> "policy": "immediate"
>>>> }
>>>>
>>>> I got this error when I tried to sync repo with the remote:
>>>> Jun 12 12:07:57 ip-1-76-158-244 rq[19389]: ssl.SSLError: [SSL] PEM lib
>>>> (_ssl.c:3503)
>>>>
>>>>
>>>> What is the correct json input for creating a remote to redhat? How do
>>>> I troubleshoot the sync issue? Thanks for your help.
>>>> _______________________________________________
>>>> Pulp-list mailing list
>>>> Pulp-list at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/pulp-list
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20190623/f01dcf87/attachment.htm>