<div dir="auto">Please file an issue at <a href="https://pulp.plan.io/issues/new/">https://pulp.plan.io/issues/new/</a> and we will try to reproduce the issue.</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jun 27, 2019, 5:50 PM Bin Li (BLOOMBERG/ 120 PARK) <<a href="mailto:bli111@bloomberg.net">bli111@bloomberg.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="m_-2986837037181417357rte-style-maintainer m_-2986837037181417357rte-pre-wrap" style="white-space:pre-wrap;font-size:small;font-family:"Courier New",Courier,"BB.FixedWidth"">Tried again for both rhel7 and rhel6 repo. The download_concurrency is always set to "1". We tried to sync rhel6 repo first and task was completed. Rerun sync on rhel7 repo failed again. Our network team didn't find any issues on the network and proxy.<div><br></div><div>This is our remote setting:</div><div> {</div><div> "_created": "2019-06-21T16:10:05.493082Z", </div><div> "_href": "/pulp/api/v3/remotes/rpm/rpm/b308a912-1dc9-46cb-b340-450a96e5ee93/", </div><div> "_last_updated": "2019-06-21T16:10:05.493107Z", </div><div> "_type": "rpm.rpm", </div><div> "download_concurrency": 1, </div><div> "name": "rhel-x86_64-server-7", </div><div> "policy": "immediate", </div><div> "proxy_url": "<a href="http://proxyhost.bloomberg.com:8000" target="_blank" rel="noreferrer">http://proxyhost.bloomberg.com:8000</a>", </div><div> "ssl_ca_certificate": "c77dce7447d4862f2b5fbd794cf6fcdc6f01e3c313cc9a12d2bf47cfd6b8db21", </div><div> "ssl_client_certificate": "c5c5dfac3e16e93b394af137ebf8f0c5eaa37277699e72dc8042a53f6133b999", </div><div> "ssl_client_key": "3f8bde4c2baf94196a9e40d0834f86a2890b3df08053eef9283b3c136aae487f", </div><div> "ssl_validation": true, </div><div> "url": "<a href="https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os/" target="_blank" rel="noreferrer">https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os/</a>", </div><div> "validate": true</div><div> }</div><div><br>Task status:</div><div><br></div><div>Allow: GET, PATCH, DELETE, HEAD, OPTIONS</div><div>Connection: close</div><div>Content-Length: 3515</div><div>Content-Type: application/json</div><div>Date: Thu, 27 Jun 2019 21:32:27 GMT</div><div>Server: gunicorn/19.9.0</div><div>Vary: Accept, Cookie</div><div>X-Frame-Options: SAMEORIGIN</div><div><br></div><div>{</div><div> "_created": "2019-06-27T18:15:28.484561Z", </div><div> "_href": "/pulp/api/v3/tasks/72da60f0-1beb-4447-b8d2-718a40b5fccc/", </div><div> "created_resources": [], </div><div> "error": {</div><div> "code": null, </div><div> "description": "Timeout on reading data from socket", </div><div> "traceback": " File \"/opt/python/3.6.5/lib/python3.6/site-packages/rq/worker.py\", line 812, in perform_job\n rv = job.perform()\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/rq/job.py\", line 588, in perform\n self._result = self._execute()\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/rq/job.py\", line 594, in _execute\n return self.func(*self.args, **self.kwargs)\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py\", line 68, in synchronize\n dv.create()\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py\", line 169, in create\n loop.run_until_complete(pipeline)\n File \"/opt/python/3.6.5/lib64/python3.6/asyncio/base_events.py\", line 468, in run_until_complete\n return future.result()\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py\", line 209, in create_pipeline\n await asyncio.gather(*futures)\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py\", line 43, in __call__\n await self.run()\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py\", line 132, in run\n pb.done += task.result() # download_count\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py\", line 155, in _handle_content_unit\n await asyncio.gather(*downloaders_for_content)\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/stages/models.py\", line 78, in download\n download_result = await downloader.run(extra_data=self.extra_data)\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/download/base.py\", line 212, in run\n return await self._run(extra_data=extra_data)\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/backoff/_async.py\", line 131, in retry\n ret = await target(*args, **kwargs)\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/download/http.py\", line 184, in _run\n to_return = await self._handle_response(response)\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/download/http.py\", line 158, in _handle_response\n chunk = await response.content.read(1048576) # 1 megabyte\n File \"/opt/python/3.6.5/lib64/python3.6/site-packages/aiohttp/streams.py\", line 369, in read\n await self._wait('read')\n File \"/opt/python/3.6.5/lib64/python3.6/site-packages/aiohttp/streams.py\", line 297, in _wait\n await waiter\n"</div><div> }, </div><div> "finished_at": "2019-06-27T21:19:18.675004Z", </div><div> "name": "pulp_rpm.app.tasks.synchronizing.synchronize", </div><div> "non_fatal_errors": [], </div><div> "parent": null, </div><div> "progress_reports": [</div><div> {</div><div> "done": 20201, </div><div> "message": "Parsed Packages", </div><div> "state": "running", </div><div> "suffix": null, </div><div> "total": 24397</div><div> }, </div><div> {</div><div> "done": 11966, </div><div> "message": "Downloading Artifacts", </div><div> "state": "failed", </div><div> "suffix": null, </div><div> "total": 11966</div><div> }, </div><div> {</div><div> "done": 5, </div><div> "message": "Downloading Metadata Files", </div><div> "state": "canceled", </div><div> "suffix": null, </div><div> "total": 5</div><div> }, </div><div> {</div><div> "done": 23300, </div><div> "message": "Associating Content", </div><div> "state": "canceled", </div><div> "suffix": null, </div><div> "total": 23300</div><div> }, </div><div> {</div><div> "done": 3583, </div><div> "message": "Parsed Erratum", </div><div> "state": "running", </div><div> "suffix": null, </div><div> "total": 3583</div><div> }</div><div> ], </div><div> "spawned_tasks": [], </div><div> "started_at": "2019-06-27T18:15:28.618096Z", </div><div> "state": "failed", </div><div> "worker": "/pulp/api/v3/workers/d1db2594-52b6-402e-8ef1-7c0a5635c3c4/"</div><div>}</div><div><br><div class="m_-2986837037181417357rte-style-maintainer" style="font-size:small;font-family:"Courier New",Courier,"BB.FixedWidth""><div><div class="m_-2986837037181417357bbg-rte-fold-content"><div class="m_-2986837037181417357bbg-rte-fold-summary">From: <a href="mailto:bbouters@redhat.com" target="_blank" rel="noreferrer">bbouters@redhat.com</a> At: 06/27/19 10:54:05</div>To: <a href="mailto:bli111@bloomberg.net" target="_blank" rel="noreferrer"> Bin Li (BLOOMBERG/ 120 PARK ) </a><br>Cc: <a href="mailto:dkliban@redhat.com" target="_blank" rel="noreferrer"> dkliban@redhat.com</a>, <a href="mailto:pulp-list@redhat.com" target="_blank" rel="noreferrer"> pulp-list@redhat.com</a><br>Subject: Re: [Pulp-list] create remote to redhat on pulp 3</div><br></div><div class="m_-2986837037181417357rte-internet-block-wrapper" style="background:white;color:black;font-family:Arial,"BB.Proportional";font-size:small;white-space:normal"><div class="m_-2986837037181417357rte-internet-block"><blockquote><div dir="ltr"><div>It says "Timeout on reading data from socket" so either this is a bug, or the remote server is not delivering you data over the socket connection, perhaps due to some rate limiting. The remote has a download_concurrency option [0] so perhaps reducing that will cause the server to give you all of the data. You can also open an issue with reproducer details and we can try to reproduce it.<br></div><div><br></div><div>[0]: <a href="https://pulp-rpm.readthedocs.io/en/latest/restapi.html#operation/remotes_rpm_rpm_create" target="_blank" rel="noreferrer">https://pulp-rpm.readthedocs.io/en/latest/restapi.html#operation/remotes_rpm_rpm_create</a></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jun 27, 2019 at 10:48 AM Bin Li (BLOOMBERG/ 120 PARK) <<a href="mailto:bli111@bloomberg.net" target="_blank" rel="noreferrer">bli111@bloomberg.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="m_-2986837037181417357gmail-m_6973067611648931688rte-style-maintainer m_-2986837037181417357gmail-m_6973067611648931688rte-pre-wrap" style="white-space:pre-wrap;font-size:small;font-family:"Courier New",Courier,"BB.FixedWidth"">I am now able to sync the redhat repo through proxy but the task failed eventually on a timeout after sync most of contents. I tried sync both rhel7 and rhel6 os repo and both gave the same error. Any idea what could cause this?<div><br></div><div><br></div><div> # ./get /pulp/api/v3/tasks/738fc1bc-463f-4d7e-9ac9-ebb1e629b6f7/</div><div>HTTP/1.1 200 OK</div><div>Allow: GET, PATCH, DELETE, HEAD, OPTIONS</div><div>Connection: close</div><div>Content-Length: 3512</div><div>Content-Type: application/json</div><div>Date: Thu, 27 Jun 2019 14:38:12 GMT</div><div>Server: gunicorn/19.9.0</div><div>Vary: Accept, Cookie</div><div>X-Frame-Options: SAMEORIGIN</div><div><br></div><div>{</div><div> "_created": "2019-06-26T20:35:38.514762Z", </div><div> "_href": "/pulp/api/v3/tasks/738fc1bc-463f-4d7e-9ac9-ebb1e629b6f7/", </div><div> "created_resources": [], </div><div> "error": {</div><div> "code": null, </div><div> "description": "Timeout on reading data from socket", </div><div> "traceback": " File \"/opt/python/3.6.5/lib/python3.6/site-packages/rq/worker.py\", line 812, in perform_job\n rv = job.perform()\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/rq/job.py\", line 588, in perform\n self._result = self._execute()\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/rq/job.py\", line 594, in _execute\n return self.func(*self.args, **self.kwargs)\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py\", line 68, in synchronize\n dv.create()\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py\", line 169, in create\n loop.run_until_complete(pipeline)\n File \"/opt/python/3.6.5/lib64/python3.6/asyncio/base_events.py\", line 468, in run_until_complete\n return future.result()\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py\", line 209, in create_pipeline\n await asyncio.gather(*futures)\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py\", line 43, in __call__\n await self.run()\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py\", line 132, in run\n pb.done += task.result() # download_count\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py\", line 155, in _handle_content_unit\n await asyncio.gather(*downloaders_for_content)\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/stages/models.py\", line 78, in download\n download_result = await downloader.run(extra_data=self.extra_data)\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/download/base.py\", line 212, in run\n return await self._run(extra_data=extra_data)\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/backoff/_async.py\", line 131, in retry\n ret = await target(*args, **kwargs)\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/download/http.py\", line 184, in _run\n to_return = await self._handle_response(response)\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/pulpcore/plugin/download/http.py\", line 158, in _handle_response\n chunk = await response.content.read(1048576) # 1 megabyte\n File \"/opt/python/3.6.5/lib64/python3.6/site-packages/aiohttp/streams.py\", line 369, in read\n await self._wait('read')\n File \"/opt/python/3.6.5/lib64/python3.6/site-packages/aiohttp/streams.py\", line 297, in _wait\n await waiter\n"</div><div> }, </div><div> "finished_at": "2019-06-26T21:57:33.499085Z", </div><div> "name": "pulp_rpm.app.tasks.synchronizing.synchronize", </div><div> "non_fatal_errors": [], </div><div> "parent": null, </div><div> "progress_reports": [</div><div> {</div><div> "done": 3757, </div><div> "message": "Downloading Artifacts", </div><div> "state": "failed", </div><div> "suffix": null, </div><div> "total": 3757</div><div> }, </div><div> {</div><div> "done": 5, </div><div> "message": "Downloading Metadata Files", </div><div> "state": "canceled", </div><div> "suffix": null, </div><div> "total": 5</div><div> }, </div><div> {</div><div> "done": 11350, </div><div> "message": "Associating Content", </div><div> "state": "canceled", </div><div> "suffix": null, </div><div> "total": 11350</div><div> }, </div><div> {</div><div> "done": 3582, </div><div> "message": "Parsed Erratum", </div><div> "state": "running", </div><div> "suffix": null, </div><div> "total": 3582</div><div> }, </div><div> {</div><div> "done": 8201, </div><div> "message": "Parsed Packages", </div><div> "state": "running", </div><div> "suffix": null, </div><div> "total": 24392</div><div> }</div><div> ], </div><div> "spawned_tasks": [], </div><div> "started_at": "2019-06-26T20:35:38.650635Z", </div><div> "state": "failed", </div><div> "worker": "/pulp/api/v3/workers/d1db2594-52b6-402e-8ef1-7c0a5635c3c4/"</div><div>}</div><div><br></div><div><br><br><div class="m_-2986837037181417357gmail-m_6973067611648931688rte-style-maintainer" style="font-size:small;font-family:"Courier New",Courier,"BB.FixedWidth""><div><div class="m_-2986837037181417357gmail-m_6973067611648931688bbg-rte-fold-content"><div class="m_-2986837037181417357gmail-m_6973067611648931688bbg-rte-fold-summary">From: <a href="mailto:dkliban@redhat.com" target="_blank" rel="noreferrer">dkliban@redhat.com</a> At: 06/23/19 07:14:50</div>To: <a href="mailto:bli111@bloomberg.net" target="_blank" rel="noreferrer"> Bin Li (BLOOMBERG/ 120 PARK ) </a><br>Cc: <a href="mailto:pulp-list@redhat.com" target="_blank" rel="noreferrer"> pulp-list@redhat.com</a><br>Subject: Re: [Pulp-list] create remote to redhat on pulp 3</div><br></div><div class="m_-2986837037181417357gmail-m_6973067611648931688rte-internet-block-wrapper" style="background:none 0% 0% repeat scroll white;color:black;font-family:Arial,"BB.Proportional";font-size:small;white-space:normal"><div class="m_-2986837037181417357gmail-m_6973067611648931688rte-internet-block"><blockquote><div dir="ltr"><div>The problem is that the proxy setting is not being respected by Pulp. I filed an issue about this[0] and also made a patch to fix it[1]. This patch should be merged in the next 24 hours.</div><div><br></div><div>If you cannot access <a href="http://cdn.redhat.com" target="_blank" rel="noreferrer">cdn.redhat.com</a> without a proxy, the error you got makes sense. I was only able to reproduce the error message when I added an incorrect entry for <a href="http://cdn.redhat.com" target="_blank" rel="noreferrer">cdn.redhat.com</a> in /etc/hosts.<br></div><div><br></div><div><br></div><div>[0] <a href="https://pulp.plan.io/issues/5011" target="_blank" rel="noreferrer">https://pulp.plan.io/issues/5011</a></div><div>[1] <a href="https://github.com/pulp/pulpcore-plugin/pull/107" target="_blank" rel="noreferrer">https://github.com/pulp/pulpcore-plugin/pull/107</a></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Jun 21, 2019 at 12:26 PM Bin Li (BLOOMBERG/ 120 PARK) <<a href="mailto:bli111@bloomberg.net" target="_blank" rel="noreferrer">bli111@bloomberg.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="m_-2986837037181417357gmail-m_6973067611648931688gmail-m_-4993253772982623711rte-style-maintainer m_-2986837037181417357gmail-m_6973067611648931688gmail-m_-4993253772982623711rte-pre-wrap" style="white-space:pre-wrap;font-size:small;font-family:"Courier New",Courier,"BB.FixedWidth"">Thanks. I am now getting a different error. Please advise<div><br></div><div> "_href": "/pulp/api/v3/tasks/fcc679b8-1ad3-4a16-834b-47b946caaeed/", </div><div> "created_resources": [], </div><div> "error": {</div><div> "code": null, </div><div> "description": "Cannot connect to host <a rel="noreferrer">cdn.redhat.com:443</a> ssl:None [Connect call failed ('23.65.16.251', 443)]", </div><div> "traceback": " File \"/opt/python/3.6.5/lib/python3.6/site-packages/rq/worker.py\", line 812, in perform_job\n rv = job.perform()\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/rq/job.py\", line 588, in perform\n self._result = self._execute()\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/rq/job.py\", line 594, in _execute\n return self.func(*self.args, **self.kwargs)\n File \"/tmp/pulp_rpm/pulp_rpm/app/tasks/synchronizing.py\", line 67, in synchronize\n dv.create()\n File \"/tmp/pulpcore-plugin/pulpcore/plugin/stages/declarative_version.py\", line 169, in create\n loop.run_until_complete(pipeline)\n File \"/opt/python/3.6.5/lib64/python3.6/asyncio/base_events.py\", line 468, in run_until_complete\n return future.result()\n File \"/tmp/pulpcore-plugin/pulpcore/plugin/stages/api.py\", line 209, in create_pipeline\n await asyncio.gather(*futures)\n File \"/tmp/pulpcore-plugin/pulpcore/plugin/stages/api.py\", line 43, in __call__\n await self.run()\n File \"/tmp/pulp_rpm/pulp_rpm/app/tasks/synchronizing.py\", line 231, in run\n result = await downloader.run()\n File \"/tmp/pulpcore-plugin/pulpcore/plugin/download/base.py\", line 212, in run\n return await self._run(extra_data=extra_data)\n File \"/opt/python/3.6.5/lib/python3.6/site-packages/backoff/_async.py\", line 131, in retry\n ret = await target(*args, **kwargs)\n File \"/tmp/pulpcore-plugin/pulpcore/plugin/download/http.py\", line 182, in _run\n async with self.session.get(self.url) as response:\n File \"/opt/python/3.6.5/lib64/python3.6/site-packages/aiohttp/client.py\", line 1005, in __aenter__\n self._resp = await self._coro\n File \"/opt/python/3.6.5/lib64/python3.6/site-packages/aiohttp/client.py\", line 476, in _request\n timeout=real_timeout\n File \"/opt/python/3.6.5/lib64/python3.6/site-packages/aiohttp/connector.py\", line 522, in connect\n proto = await self._create_connection(req, traces, timeout)\n File \"/opt/python/3.6.5/lib64/python3.6/site-packages/aiohttp/connector.py\", line 854, in _create_connection\n req, traces, timeout)\n File \"/opt/python/3.6.5/lib64/python3.6/site-packages/aiohttp/connector.py\", line 992, in _create_direct_connection\n raise last_exc\n File \"/opt/python/3.6.5/lib64/python3.6/site-packages/aiohttp/connector.py\", line 974, in _create_direct_connection\n req=req, client_error=client_error)\n File \"/opt/python/3.6.5/lib64/python3.6/site-packages/aiohttp/connector.py\", line 931, in _wrap_create_connection\n raise client_error(req.connection_key, exc) from exc\n"</div><div> }, </div><div><br></div><div><br></div><div>I was able to connect using client cert through proxy</div><div> # curl -Iv --cacert /etc/rhsm/ca/redhat-uep.pem --key /etc/pki/entitlement/8521692907269500331-key.pem --cert /etc/pki/entitlement/8521692907269500331.pem <a href="https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os/" target="_blank" rel="noreferrer">https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os/</a></div><div>* About to connect() to proxy ... port ... (#0)</div><div>* Trying ...</div><div>* Connected to ... () port ... (#0)</div><div>* Establish HTTP proxy tunnel to <a rel="noreferrer">cdn.redhat.com:443</a></div><div>> CONNECT <a rel="noreferrer">cdn.redhat.com:443</a> HTTP/1.1</div><div>> Host: <a rel="noreferrer">cdn.redhat.com:443</a></div><div>> User-Agent: curl/7.29.0</div><div>> Proxy-Connection: Keep-Alive</div><div>> </div><div>< HTTP/1.1 200 Connection established</div><div>HTTP/1.1 200 Connection established</div><div>< </div><div><br></div><div>* Proxy replied OK to CONNECT request</div><div>* Initializing NSS with certpath: sql:/etc/pki/nssdb</div><div><br><br><div class="m_-2986837037181417357gmail-m_6973067611648931688gmail-m_-4993253772982623711rte-style-maintainer" style="font-size:small;font-family:"Courier New",Courier,"BB.FixedWidth""><div><div class="m_-2986837037181417357gmail-m_6973067611648931688gmail-m_-4993253772982623711bbg-rte-fold-content"><div class="m_-2986837037181417357gmail-m_6973067611648931688gmail-m_-4993253772982623711bbg-rte-fold-summary">From: <a href="mailto:dkliban@redhat.com" target="_blank" rel="noreferrer">dkliban@redhat.com</a> At: 06/20/19 16:34:40</div>To: <a href="mailto:bli111@bloomberg.net" target="_blank" rel="noreferrer"> Bin Li (BLOOMBERG/ 120 PARK ) </a><br>Cc: <a href="mailto:pulp-list@redhat.com" target="_blank" rel="noreferrer"> pulp-list@redhat.com</a><br>Subject: Re: [Pulp-list] create remote to redhat on pulp 3</div><br></div><div class="m_-2986837037181417357gmail-m_6973067611648931688gmail-m_-4993253772982623711rte-internet-block-wrapper" style="background:none 0% 0% repeat scroll white;color:black;font-family:Arial,"BB.Proportional";font-size:small;white-space:normal"><div class="m_-2986837037181417357gmail-m_6973067611648931688gmail-m_-4993253772982623711rte-internet-block"><blockquote><div dir="ltr">You need to make sure that the following services are running: pulp-resource-manager, pulp-worker@0, pulp-worker@1 (at least 1 worker is needed). <br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jun 20, 2019 at 4:11 PM Bin Li (BLOOMBERG/ 120 PARK) <<a href="mailto:bli111@bloomberg.net" target="_blank" rel="noreferrer">bli111@bloomberg.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="m_-2986837037181417357gmail-m_6973067611648931688gmail-m_-4993253772982623711gmail-m_6618610937059462134rte-style-maintainer m_-2986837037181417357gmail-m_6973067611648931688gmail-m_-4993253772982623711gmail-m_6618610937059462134rte-pre-wrap" style="white-space:pre-wrap;font-size:small;font-family:"Courier New",Courier,"BB.FixedWidth"">I installed the latest version of component. No more errors when try to sync the repo but the task is in waiting state forever. Any idea what I should check?<div><br></div><div> "versions": [</div><div> {</div><div> "component": "pulpcore", </div><div> "version": "3.0.0rc3.dev0"</div><div> }, </div><div> {</div><div> "component": "pulpcore-plugin", </div><div> "version": "0.1.0rc3.dev0"</div><div> }, </div><div> {</div><div> "component": "pulp_rpm", </div><div> "version": "3.0.0b4.dev0"</div><div> }</div><div><br><div><br><br><div class="m_-2986837037181417357gmail-m_6973067611648931688gmail-m_-4993253772982623711gmail-m_6618610937059462134rte-style-maintainer" style="font-size:small;font-family:"Courier New",Courier,"BB.FixedWidth""><div><div class="m_-2986837037181417357gmail-m_6973067611648931688gmail-m_-4993253772982623711gmail-m_6618610937059462134bbg-rte-fold-content"><div class="m_-2986837037181417357gmail-m_6973067611648931688gmail-m_-4993253772982623711gmail-m_6618610937059462134bbg-rte-fold-summary">From: <a href="mailto:dkliban@redhat.com" target="_blank" rel="noreferrer">dkliban@redhat.com</a> At: 06/18/19 13:42:01</div>To: <a href="mailto:bli111@bloomberg.net" target="_blank" rel="noreferrer"> Bin Li (BLOOMBERG/ 120 PARK ) </a><br>Cc: <a href="mailto:pulp-list@redhat.com" target="_blank" rel="noreferrer"> pulp-list@redhat.com</a><br>Subject: Re: [Pulp-list] create remote to redhat on pulp 3</div><br></div><div class="m_-2986837037181417357gmail-m_6973067611648931688gmail-m_-4993253772982623711gmail-m_6618610937059462134rte-internet-block-wrapper" style="background:none 0% 0% repeat scroll white;color:black;font-family:Arial,"BB.Proportional";font-size:small;white-space:normal"><div class="m_-2986837037181417357gmail-m_6973067611648931688gmail-m_-4993253772982623711gmail-m_6618610937059462134rte-internet-block"><blockquote><div dir="ltr"><div>This issue has now been resolved and a fix is available on master branch of pulpcore-plugin[0]. </div><div><br></div><div><br></div><div>[0] <a href="https://github.com/pulp/pulpcore-plugin/tree/master/" target="_blank" rel="noreferrer">https://github.com/pulp/pulpcore-plugin/tree/master/</a></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jun 12, 2019 at 1:51 PM Dennis Kliban <<a href="mailto:dkliban@redhat.com" target="_blank" rel="noreferrer">dkliban@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>There is an open issue about this[0]. I recommend clicking the watch button so you get notifications about it. We should be fixing this soon. <br></div><div><br></div><div>[0] <a href="https://pulp.plan.io/issues/4825" target="_blank" rel="noreferrer">https://pulp.plan.io/issues/4825</a></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jun 12, 2019 at 1:42 PM Bin Li (BLOOMBERG/ 120 PARK) <<a href="mailto:bli111@bloomberg.net" target="_blank" rel="noreferrer">bli111@bloomberg.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="m_-2986837037181417357gmail-m_6973067611648931688gmail-m_-4993253772982623711gmail-m_6618610937059462134gmail-m_-1423717654734810632gmail-m_-5849831393299595243rte-style-maintainer m_-2986837037181417357gmail-m_6973067611648931688gmail-m_-4993253772982623711gmail-m_6618610937059462134gmail-m_-1423717654734810632gmail-m_-5849831393299595243rte-pre-wrap" style="white-space:pre-wrap;font-size:small;font-family:"Courier New",Courier,"BB.FixedWidth""><br><div class="m_-2986837037181417357gmail-m_6973067611648931688gmail-m_-4993253772982623711gmail-m_6618610937059462134gmail-m_-1423717654734810632gmail-m_-5849831393299595243rte-style-maintainer m_-2986837037181417357gmail-m_6973067611648931688gmail-m_-4993253772982623711gmail-m_6618610937059462134gmail-m_-1423717654734810632gmail-m_-5849831393299595243rte-pre-wrap" style="white-space:pre-wrap;font-size:small;font-family:"Courier New",Courier,"BB.FixedWidth"">I was not able to sync a protected repo from redhat. The host running pulp 3 subscribes to redhat directly.<div><br></div><div>I first tried to use the following to create remote:<div>{</div><div>"name": "rhel-x86_64-server-7",</div><div>"url": "<a href="https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os" target="_blank" rel="noreferrer">https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os</a>",</div><div>"ssl_ca_certificate": "/etc/rhsm/ca/redhat-uep.pem",</div><div>"ssl_client_certificate": "/etc/pki/consumer/cert.pem", </div><div>"ssl_client_key": "/etc/pki/consumer/key.pem",</div><div>"ssl_validation": true,</div><div>"proxy_url": "<a href="http://proxy:80" target="_blank" rel="noreferrer">http://proxy:80</a>",</div><div>"download_concurrency": 1,</div><div>"policy": "immediate"</div><div>} </div><div><br></div><div>Remote was created but I got errors "Jun 12 12:51:43 ip-1-76-158-244 rq[19389]: pulp: rq.worker:ERROR: ssl.SSLError: [PEM: NO_START_LINE] no start line (_ssl.c:3626)" when I tried to sync with http POST :24817${REMOTE_HREF}sync/ repository=$REPO_HREF</div><div><br></div><div>Then I tried to converted the pem file to a json format </div><div>{</div><div>"name": "rhel-x86_64-server-7",</div><div>"url": "<a href="https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os" target="_blank" rel="noreferrer">https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os</a>",</div><div>"ssl_ca_certificate": "-----BEGIN CERTIFICATE-----\nMIIG/TCCBOWgAwIBAgIBNzANBgkqhkiG9w0BAQUFADCBsTELMAkGA1UEBhMCVVMx\nFzAVBg</div><div>...</div><div>"ssl_client_certificate": "-----BEGIN CERTIFICATE-----\nMIIMojCCCoqgAwIBAgIISqmnKnJ9sEowDQ</div><div>...</div><div>"ssl_client_key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKQIBAAKCAgEAuujl5HxnIDTSOemZOyH3Jr5xxgrc1rCee</div><div>...</div><div>"ssl_validation": true,</div><div>"proxy_url": "<a href="http://proxy:80" target="_blank" rel="noreferrer">http://proxy:80</a>",</div><div>"download_concurrency": 1,</div><div>"policy": "immediate"</div><div>} </div><div><br></div><div>I got this error when I tried to sync repo with the remote:</div><div>Jun 12 12:07:57 ip-1-76-158-244 rq[19389]: ssl.SSLError: [SSL] PEM lib (_ssl.c:3503)</div><div><br></div></div><div><br></div><div>What is the correct json input for creating a remote to redhat? How do I troubleshoot the sync issue? Thanks for your help.</div></div></div></div>_______________________________________________<br>Pulp-list mailing list<br><a href="mailto:Pulp-list@redhat.com" target="_blank" rel="noreferrer">Pulp-list@redhat.com</a><br><a href="https://www.redhat.com/mailman/listinfo/pulp-list" target="_blank" rel="noreferrer">https://www.redhat.com/mailman/listinfo/pulp-list</a></blockquote></div></blockquote></div></blockquote><br></div></div></div></div></div></div></div></blockquote></div></blockquote><br></div></div></div></div></div></div></blockquote></div></blockquote><br></div></div></div></div></div></div>_______________________________________________<br>Pulp-list mailing list<br><a href="mailto:Pulp-list@redhat.com" target="_blank" rel="noreferrer">Pulp-list@redhat.com</a><br><a href="https://www.redhat.com/mailman/listinfo/pulp-list" target="_blank" rel="noreferrer">https://www.redhat.com/mailman/listinfo/pulp-list</a></blockquote></div></blockquote><br></div></div></div></div></div></div></blockquote></div>