<div dir="ltr">This looks like a problem with the client certificate. The very first request for a file that pulp tries to download is receiving a 403 response from the CDN. Please make sure you have the correct client certificate and client key configured. <br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Mar 17, 2020 at 5:53 PM Venkataramana Bora <<a href="mailto:venkbora@in.ibm.com">venkbora@in.ibm.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" style="font-family:Arial,Helvetica,sans-serif;font-size:10pt"><div dir="ltr"> </div>
<div dir="ltr"> </div>
<div dir="ltr"><div>Hi Teamn ,<br>Getting errors as shown here when we are trying to sync RHEL 8 repos (Base OS and Appstream) creatd on Pulp master. The same Pulp master we already using for RHEL 6 and 7 repos with out any issues. Strugling with RHEL 8 repos creation,<br>1.Could you please let us know whethere RHEL 8 rpm repos sync works on Pulp master version 2.16 or not ?<br>2.feed_ca_cert "redhat-uep.pem" and feed_cert "rhel-identity.pem" are there . We have no issues with RHEL 6 and 7 repo syncs, those created some years ago and working well.<br> Is there any thing specifically need to do for RHEL 8 repos in terms of ca cert/feed cert ? We see "failed with code 403 Forbiddenin" in var log messages.</div>
<div>3.pulp-admin tasks details Traceback shows "/usr/lib/python2.7/..."</div>
<div> For this added these 3 lines in /etc/pulp/server/plugins.conf.d/yum_importer.json as recommended here <a href="https://pulp.plan.io/issues/6327" target="_blank">https://pulp.plan.io/issues/6327</a></div>
<div> {<br> "validate":true<br>}</div>
<div>After adding that json , I see that RHEL 8 repodata xml.gz in rhel8/x86_64/baseos/os/repodata/ but not "packages" folder created yet , checked that thru pulp web access .<br>Please help resolving this issue .</div>
<div><br>Feed url: <a href="https://cdn.redhat.com/content/dist/rhel8/$releasever/x86_64/baseos/os" target="_blank">https://cdn.redhat.com/content/dist/rhel8/$releasever/x86_64/baseos/os</a><br>Feed url: <a href="https://cdn.redhat.com/content/dist/rhel8/$releasever/x86_64/appstream/os" target="_blank">https://cdn.redhat.com/content/dist/rhel8/$releasever/x86_64/appstream/os</a><br>Repoid for these urls are :<br>rhel-8-server-baseos-x86_64<br>rhel-8-server-appstream-x86_64</div>
<div>Note: Tried with "8" in place "$releasever" in feed but same error . Our Pulp master ver.2.16 is on CentOS 7.7.</div>
<div>------------------------------------------------------------------------------------------------------------------------------------------------------------------------------</div>
<div>Tasks performed:</div>
<div> </div>
<div><div>[root@swy01opplppr01 ~]# pulp-admin rpm repo sync run --force-full --repo-id rhel-8-server-baseos-x86_64<br>+----------------------------------------------------------------------+<br> Synchronizing Repository [rhel-8-server-baseos-x86_64]<br>+----------------------------------------------------------------------+</div>
<div>The following sync configuration options will be used:</div>
<div>Force Full: True</div>
<div><br>This command may be exited via ctrl+c without affecting the request.</div>
<div><br>Downloading metadata...<br>[\]<br>... completed</div>
<div>Downloading repository content...<br>[-]<br>[==================================================] 100%<br>RPMs: 0/0 items<br>Delta RPMs: 0/0 items</div>
<div>... completed</div>
<div>Downloading distribution files...<br>[==================================================] 100%<br>Distributions: 0/0 items</div>
<div>Task Failed</div>
<div>Error retrieving metadata: Forbidden</div>
<div><br>-------------------------------------------------------------------------------------------------------------------------<br>From /var/log/messages:</div>
<div><br>Mar 17 17:11:23 swy01opplppr01 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[c5b01662-a20a-4c44-b114-b1ba595f05fc]<br>Mar 17 17:11:23 swy01opplppr01 pulp: celery.app.trace:INFO: [55ba0954] Task pulp.server.async.tasks._queue_reserved_task[55ba0954-1a18-410a-9507-f61d74c4776f] succeeded in 0.115055091001s: None<br>Mar 17 17:11:23 swy01opplppr01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [fbbc36d3] Downloading metadata from <a href="https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/" target="_blank">https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/</a>.<br>Mar 17 17:11:23 swy01opplppr01 pulp: urllib3.connectionpool:INFO: Starting new HTTPS connection (1): <a href="http://cdn.redhat.com" target="_blank">cdn.redhat.com</a><br>Mar 17 17:11:23 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: Download failed: Download of <a href="https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/repodata/repomd.xml" target="_blank">https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/repodata/repomd.xml</a> failed with code 403: Forbidden<br>Mar 17 17:11:24 swy01opplppr01 pulp: urllib3.connectionpool:INFO: [fbbc36d3] Starting new HTTPS connection (1): <a href="http://cdn.redhat.com" target="_blank">cdn.redhat.com</a><br>Mar 17 17:11:24 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: [fbbc36d3] Download failed: Download of <a href="https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os" target="_blank">https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os</a> failed with code 403: Forbidden<br>Mar 17 17:11:24 swy01opplppr01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [fbbc36d3] Downloading metadata from <a href="https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/" target="_blank">https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/</a>.<br>Mar 17 17:11:24 swy01opplppr01 pulp: urllib3.connectionpool:INFO: Starting new HTTPS connection (1): <a href="http://cdn.redhat.com" target="_blank">cdn.redhat.com</a><br>Mar 17 17:11:24 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: Download failed: Download of <a href="https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/repodata/repomd.xml" target="_blank">https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/repodata/repomd.xml</a> failed with code 403: Forbidden<br>Mar 17 17:11:25 swy01opplppr01 pulp: urllib3.connectionpool:INFO: [fbbc36d3] Starting new HTTPS connection (1): <a href="http://cdn.redhat.com" target="_blank">cdn.redhat.com</a><br>Mar 17 17:11:25 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: [fbbc36d3] Download failed: Download of <a href="https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os" target="_blank">https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os</a> failed with code 403: Forbidden<br>Mar 17 17:11:25 swy01opplppr01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [fbbc36d3] Downloading metadata from <a href="https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/" target="_blank">https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/</a>.<br>Mar 17 17:11:25 swy01opplppr01 pulp: urllib3.connectionpool:INFO: Starting new HTTPS connection (1): <a href="http://cdn.redhat.com" target="_blank">cdn.redhat.com</a><br>Mar 17 17:11:25 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: Download failed: Download of <a href="https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/repodata/repomd.xml" target="_blank">https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/repodata/repomd.xml</a> failed with code 403: Forbidden<br>Mar 17 17:11:26 swy01opplppr01 pulp: urllib3.connectionpool:INFO: [fbbc36d3] Starting new HTTPS connection (1): <a href="http://cdn.redhat.com" target="_blank">cdn.redhat.com</a><br>Mar 17 17:11:26 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: [fbbc36d3] Download failed: Download of <a href="https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os" target="_blank">https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os</a> failed with code 403: Forbidden<br>Mar 17 17:11:26 swy01opplppr01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [fbbc36d3] Downloading metadata from <a href="https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/" target="_blank">https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/</a>.<br>Mar 17 17:11:26 swy01opplppr01 pulp: urllib3.connectionpool:INFO: Starting new HTTPS connection (1): <a href="http://cdn.redhat.com" target="_blank">cdn.redhat.com</a><br>Mar 17 17:11:27 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: Download failed: Download of <a href="https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/repodata/repomd.xml" target="_blank">https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/repodata/repomd.xml</a> failed with code 403: Forbidden<br>Mar 17 17:11:27 swy01opplppr01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [fbbc36d3] Downloading additional units.<br>Mar 17 17:11:27 swy01opplppr01 pulp: urllib3.connectionpool:INFO: Starting new HTTPS connection (1): <a href="http://cdn.redhat.com" target="_blank">cdn.redhat.com</a><br>Mar 17 17:11:28 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: Download failed: Download of <a href="https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/.treeinfo" target="_blank">https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/.treeinfo</a> failed with code 404: Not Found<br>Mar 17 17:11:28 swy01opplppr01 pulp: urllib3.connectionpool:INFO: Starting new HTTPS connection (1): <a href="http://cdn.redhat.com" target="_blank">cdn.redhat.com</a><br>Mar 17 17:11:29 swy01opplppr01 pulp: nectar.downloaders.threaded:INFO: Download failed: Download of <a href="https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/treeinfo" target="_blank">https://cdn.redhat.com/content/dist/rhel8//x86_64/baseos/os/treeinfo</a> failed with code 404: Not Found<br>Mar 17 17:11:30 swy01opplppr01 pulp: pulp.server.async.tasks:INFO: [fbbc36d3] Task failed : [fbbc36d3-fddd-4f54-9be6-770acecf374f] : Error retrieving metadata: Forbidden<br>Mar 17 17:11:30 swy01opplppr01 pulp: celery.app.trace:INFO: [fbbc36d3] Task pulp.server.managers.repo.sync.sync[fbbc36d3-fddd-4f54-9be6-770acecf374f] raised expected: PulpCodedException()<br>Mar 17 17:11:30 swy01opplppr01 pulp: celery.app.trace:INFO: [c5b01662] Task pulp.server.async.tasks._release_resource[c5b01662-a20a-4c44-b114-b1ba595f05fc] succeeded in 0.00344043700079s: None</div>
<div>-----------------------------------------------------------------------------------------------------</div>
<div> </div>
<div>[root@swy01opplppr01 ~]# pulp-admin tasks details --task-id fbbc36d3-fddd-4f54-9be6-770acecf374f<br>+----------------------------------------------------------------------+<br> Task Details<br>+----------------------------------------------------------------------+</div>
<div>Operations: sync<br>Resources: rhel-8-server-baseos-x86_64 (repository)<br>State: Failed<br>Start Time: 2020-03-17T21:11:23Z<br>Finish Time: 2020-03-17T21:11:30Z<br>Result: N/A<br>Task Id: fbbc36d3-fddd-4f54-9be6-770acecf374f<br>Worker Name: reserved_resource_worker-3@swy01opplppr01.ops.hosting.knxa<br>Progress Report:<br> Yum Importer:<br> Comps:<br> State: FINISHED<br> Content:<br> Details:<br> Drpm Done: 0<br> Drpm Total: 0<br> Rpm Done: 0<br> Rpm Total: 0<br> Error Details:<br> Items Left: 0<br> Items Total: 0<br> Size Left: 0<br> Size Total: 0<br> State: FINISHED<br> Distribution:<br> Error Details:<br> Items Left: 0<br> Items Total: 0<br> State: FINISHED<br> Errata:<br> State: FINISHED<br> Metadata:<br> State: FINISHED<br> Purge Duplicates:<br> State: FINISHED<br>Traceback: Traceback (most recent call last): File<br> "/usr/lib/python2.7/site-packages/celery/app/trace.py", line<br> 382, in trace_task R = retval = fun(*args, **kwargs)<br> File<br> "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py",<br> line 529, in __call__ return super(Task,<br> self).__call__(*args, **kwargs) File<br> "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py",<br> line 107, in __call__ return super(PulpTask,<br> self).__call__(*args, **kwargs) File<br> "/usr/lib/python2.7/site-packages/celery/app/trace.py", line<br> 641, in __protected_call__ return self.run(*args,<br> **kwargs) File<br> "/usr/lib/python2.7/site-packages/pulp/server/controllers/repo<br> sitory.py", line 769, in sync sync_report =<br> sync_repo(transfer_repo, conduit, call_config) File<br> "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py",<br> line 737, in wrap_f return f(*args, **kwargs) File<br> "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/y<br> um/importer.py", line 82, in sync_repo report =<br> self._current_sync.run() File<br> "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/y<br> um/sync.py", line 335, in run<br> reason=self.repomd_not_found_reason) PulpCodedException: Error<br> retrieving metadata: Forbidden</div></div>
<div> </div>
<div>-------------------------------------------------------------------------------<br><br>Sincerely,<br>Ramana Bora</div></div></div><br>
_______________________________________________<br>
Pulp-list mailing list<br>
<a href="mailto:Pulp-list@redhat.com" target="_blank">Pulp-list@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/pulp-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a></blockquote></div>