[Pulp-list] Pulp RHEL Repo Download Forbidden??
Gavin Jones
gavinj84 at gmail.com
Mon May 18 03:54:48 UTC 2015
Ok if it helps anyone I have deleted the repos in Pulp and created them
again it's all fine now.
After checking on my servers which were directly connected to RHN there
certificates had been updated, hence the break in the PULP sync.
Thanks
On Wed, May 6, 2015 at 9:26 AM, Gavin Jones <gavinj84 at gmail.com> wrote:
> hey Josh / Reece,
>
> I hate to say I am glad, I am not the only one with this issue. Did anyone
> on #Pulp speak about the issue?
>
> Let us know how you go with troubleshooting this.
>
>
> Thanks
>
>
> On Wed, May 6, 2015 at 3:22 AM, Webb, Reece <Reece.Webb at ucsf.edu> wrote:
>
>> I have seen this issue for months, a sync fails 9 times out of 10. It
>> appears to be an issue (for me at least) on the Redhat side of things. I
>> use curl to get more info.
>>
>> I’ll run it one time and get a failure:
>>
>> # curl -v —key ./Workstation-Entitlement.pem --cert
>> ./Workstation-Entitlement.pem -k
>> https://cdn.redhat.com/content/dist/rhel/workstation/7/7Workstation/x86_64/kickstart/treeinfo
>>
>> - About to connect() to cdn.redhat.com port 443 (#0)
>> * Trying 184.84.192.251...
>> * Connected to cdn.redhat.com (184.84.192.251) port 443 (#0)
>> * Initializing NSS with certpath: sql:/etc/pki/nssdb
>> * skipping SSL peer certificate verification
>> * NSS: client certificate from file
>> * subject: CN=8a85f9894bd9c252014be203f1a6096f
>> * start date: Aug 01 04:00:00 2014 GMT
>> * expire date: Aug 01 03:59:59 2015 GMT
>> * common name: 8a85f9894bd9c252014be203f1a6096f
>> * issuer: E=ca-support at redhat.com,CN=Red Hat Candlepin
>> Authority,OU=Red Hat Network,O="Red Hat, Inc.",ST=North Carolina,C=US
>> * SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA
>> * Server certificate:
>> * subject: CN=cdn.redhat.com,OU=Red Hat Network,O=Red
>> Hat,L=Raleigh,ST=North Carolina,C=US
>> * start date: May 14 19:48:02 2014 GMT
>> * expire date: May 11 19:48:02 2024 GMT
>> * common name: cdn.redhat.com
>> * issuer: E=ca-support at redhat.com,CN=Red Hat Entitlement Operations
>> Authority,OU=Red Hat Network,O="Red Hat, Inc.",ST=North Carolina,C=US
>> > GET
>> /content/dist/rhel/workstation/7/7Workstation/x86_64/kickstart/treeinfo
>> HTTP/1.1
>> > User-Agent: curl/7.29.0
>> > Host: cdn.redhat.com
>> > Accept: */*
>> >
>> < HTTP/1.1 403 Forbidden
>> < Server: AkamaiGHost
>> < Mime-Version: 1.0
>> < Content-Type: text/html
>> < Content-Length: 369
>> < Expires: Tue, 05 May 2015 17:13:05 GMT
>> < Date: Tue, 05 May 2015 17:13:05 GMT
>> < X-Cache: TCP_DENIED from
>> a128-241-218-165.deploy.akamaitechnologies.com
>> (AkamaiGHost/7.2.0-15182023) (-)
>> < Connection: keep-alive
>> < EJ-HOST: edgejavaapp2.prod.a4.vary.redhat.com
>> < X-Akamai-Request-ID: 4a217f0
>> <
>> <HTML><HEAD>
>> <TITLE>Access Denied</TITLE>
>> </HEAD><BODY>
>> <H1>Access Denied</H1>
>>
>> You don't have permission to access
>> "http://cdn.redhat.com/content/dist/rhel/workstation/7/7Workstation/x86_64/kickstart/treeinfo"
>> on this server.<P>
>> Reference #18.a5daf180.1430845985.4a217f0
>>
>>
>>
>> And then I’ll re-run the command seconds later with a successful
>> response:
>>
>> # curl -v --key ./Workstation-Entitlement.pem --cert
>> ./Workstation-Entitlement.pem -k
>> https://cdn.redhat.com/content/dist/rhel/workstation/7/7Workstation/x86_64/kickstart/treeinfo
>> * About to connect() to cdn.redhat.com port 443 (#0)
>> * Trying 184.84.192.251...
>> * Connected to cdn.redhat.com (184.84.192.251) port 443 (#0)
>> * Initializing NSS with certpath: sql:/etc/pki/nssdb
>> * skipping SSL peer certificate verification
>> * NSS: client certificate from file
>> * subject: CN=8a85f9894bd9c252014be203f1a6096f
>> * start date: Aug 01 04:00:00 2014 GMT
>> * expire date: Aug 01 03:59:59 2015 GMT
>> * common name: 8a85f9894bd9c252014be203f1a6096f
>> * issuer: E=ca-support at redhat.com,CN=Red Hat Candlepin Authority,OU=Red
>> Hat Network,O="Red Hat, Inc.",ST=North Carolina,C=US
>> * SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA
>> * Server certificate:
>> * subject: CN=cdn.redhat.com,OU=Red Hat Network,O=Red
>> Hat,L=Raleigh,ST=North Carolina,C=US
>> * start date: May 14 19:48:02 2014 GMT
>> * expire date: May 11 19:48:02 2024 GMT
>> * common name: cdn.redhat.com
>> * issuer: E=ca-support at redhat.com,CN=Red Hat Entitlement Operations
>> Authority,OU=Red Hat Network,O="Red Hat, Inc.",ST=North Carolina,C=US
>> > GET
>> /content/dist/rhel/workstation/7/7Workstation/x86_64/kickstart/treeinfo
>> HTTP/1.1
>> > User-Agent: curl/7.29.0
>> > Host: cdn.redhat.com
>> > Accept: */*
>> >
>> < HTTP/1.1 200 OK
>> < Server: Apache
>> < ETag: "11f6fa6eaa857d424b630447ab5334de:1424446169"
>> < Last-Modified: Fri, 20 Feb 2015 08:29:44 GMT
>> < Accept-Ranges: bytes
>> < Content-Length: 1471
>> < Content-Type: text/plain
>> < Date: Tue, 05 May 2015 17:16:10 GMT
>> < X-Cache: TCP_HIT from a128-241-218-165.deploy.akamaitechnologies.com
>> (AkamaiGHost/7.2.0-15182023) (-)
>> < Connection: keep-alive
>> < EJ-HOST: rhej03.web.prod.ext.phx2.redhat.com
>> < X-Akamai-Request-ID: 4a57fb3
>> <
>> [checksums]
>> LiveOS/squashfs.img =
>> sha256:198ef91d868e76c994680645964ef3873ec66fddb84be450370b051facaec8aa
>> images/pxeboot/initrd.img =
>> sha256:101b3b5630b7032557be95aa8dcef50b01d8bfcdfa33429cea30fe09eaae9426
>> images/pxeboot/upgrade.img =
>> sha256:03453b1f504e548ab9a933daa2f1fd440e48638f5deb9fac50be7dad929c1907
>> images/pxeboot/vmlinuz =
>> sha256:67421a4877919ff0c16c27a53cba229e5f0771ae9cd32f3918caae2124a5a710
>> repodata/repomd.xml =
>> sha256:014184dc5e503979a5577a97423e4340e5f71ac2746250bbdce91e0301b8c93f
>>
>> …
>>
>>
>> I never have this issue syncing the Server repositories, only
>> Workstation (and RHEL5 Client).
>>
>> Reece
>>
>>
>>
>> From: "Baird, Josh"
>> Date: Tuesday, May 5, 2015 at 4:23 AM
>> To: Gavin Jones, "pulp-list at redhat.com"
>> Subject: Re: [Pulp-list] Pulp RHEL Repo Download Forbidden??
>>
>> Hi Gavin,
>>
>>
>>
>> I am having the same problem. I just noticed that it was occurring
>> yesterday. I re-issued new entitlement certificates with valid expiration
>> dates from RHN and the problem is still occurring. I have verified that my
>> certificates contain path/entitlements for the channels that I am trying to
>> sync (via rct cat-cert). Occasionally, Pulp will be able to download the
>> metadata for certain channels, but then get 'Forbidden' when downloading
>> individual packages. Other times, it will throw a 'Forbidden' before being
>> able to download the metadata as you pasted below.
>>
>>
>>
>> I am going to hopefully spend some time working with the developers in
>> #pulp today to get this figured out. I have a feeling it is CDN related,
>> but I'm not exactly sure at this point.
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Josh
>>
>>
>>
>> *From:* pulp-list-bounces at redhat.com [mailto:pulp-list-bounces at redhat.com
>> <pulp-list-bounces at redhat.com>] *On Behalf Of *Gavin Jones
>> *Sent:* Tuesday, May 05, 2015 12:13 AM
>> *To:* pulp-list at redhat.com
>> *Subject:* [Pulp-list] Pulp RHEL Repo Download Forbidden??
>>
>>
>>
>>
>>
>>
>>
>> Hi Everyone, I seem to be getting an error when downloading from the
>> Redhat Repos. This has only just stopped working and has been working fine
>> for months.
>>
>>
>>
>> It looks to be certificate related I believe from the logs.
>>
>>
>>
>>
>>
>> * Firstly I have not changed anything on the pulp side
>>
>> * I have checked my subscriptions are still active and the hosts that are
>> connected to RHEL are still connected.
>>
>>
>>
>>
>>
>> - Pulp Version:
>>
>>
>>
>> rpm -qa | grep -i pulp
>>
>>
>>
>> python-pulp-client-lib-2.6.0-1.el7.noarch
>>
>> pulp-rpm-plugins-2.6.0-1.el7.noarch
>>
>> python-pulp-bindings-2.6.0-1.el7.noarch
>>
>> python-kombu-3.0.24-5.pulp.el7.noarch
>>
>> python-isodate-0.5.0-4.pulp.el7.noarch
>>
>> pulp-admin-client-2.6.0-1.el7.noarch
>>
>> pulp-rpm-admin-extensions-2.6.0-1.el7.noarch
>>
>> python-pulp-common-2.6.0-1.el7.noarch
>>
>> pulp-server-2.6.0-1.el7.noarch
>>
>> pulp-selinux-2.6.0-1.el7.noarch
>>
>> python-pulp-rpm-common-2.6.0-1.el7.noarch
>>
>>
>>
>>
>>
>> - Attempting to download the repo.
>>
>>
>>
>> Please see below:
>>
>>
>>
>> pulp-admin rpm repo sync run --repo-id=rhel-7-server-rhn-tools-rpms
>>
>> +----------------------------------------------------------------------+
>>
>> Synchronizing Repository [rhel-7-server-rhn-tools-rpms]
>>
>> +----------------------------------------------------------------------+
>>
>>
>>
>> This command may be exited via ctrl+c without affecting the request.
>>
>>
>>
>>
>>
>> Downloading metadata...
>>
>> [\]
>>
>> ... failed
>>
>>
>>
>> Forbidden
>>
>>
>>
>>
>>
>> Task Failed
>>
>>
>>
>> Importer indicated a failed response
>>
>>
>>
>>
>>
>>
>>
>> - Error Log
>>
>>
>>
>> journalctl -f
>>
>>
>>
>> ay 05 13:33:05 pulp01.rap.local pulp[2741]:
>> pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata from
>> https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/rhn-tools/os/
>> .
>>
>> May 05 13:33:05 pulp01.rap.local pulp[2741]:
>> requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS
>> connection (1): cdn.redhat.com
>>
>> May 05 13:33:06 pulp01.rap.local pulp[2741]:
>> pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) sync failed
>>
>> May 05 13:33:06 pulp01.rap.local pulp[2741]:
>> pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) Traceback (most
>> recent call last):
>>
>> May 05 13:33:06 pulp01.rap.local pulp[2741]:
>> pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) File
>> "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py",...e
>> 104, in run
>>
>> May 05 13:33:06 pulp01.rap.local pulp[2741]:
>> pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) metadata_files
>> = self.get_metadata()
>>
>> May 05 13:33:06 pulp01.rap.local pulp[2741]:
>> pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) File
>> "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py",...
>> get_metadata
>>
>> May 05 13:33:06 pulp01.rap.local pulp[2741]:
>> pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) raise
>> FailedException(str(e))
>>
>> May 05 13:33:06 pulp01.rap.local pulp[2741]:
>> pulp_rpm.plugins.importers.yum.sync:ERROR: (2741-28000) FailedException:
>> Forbidden
>>
>> May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR:
>> (2554-28000) Task
>> pulp.server.managers.repo.sync.sync[81644b21-6bec-47dd-a31b-552baa2a27a8]
>> raised unexpected: P...d response',)
>>
>> May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR:
>> (2554-28000) Traceback (most recent call last):
>>
>> May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR:
>> (2554-28000) File "/usr/lib/python2.7/site-packages/celery/app/trace.py",
>> line 240, in trace_task
>>
>> May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR:
>> (2554-28000) R = retval = fun(*args, **kwargs)
>>
>> May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR:
>> (2554-28000) File
>> "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 328, in
>> __call__
>>
>> May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR:
>> (2554-28000) return super(Task, self).__call__(*args, **kwargs)
>>
>> May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR:
>> (2554-28000) File "/usr/lib/python2.7/site-packages/celery/app/trace.py",
>> line 437, in __protected_call__
>>
>> May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR:
>> (2554-28000) return self.run(*args, **kwargs)
>>
>> May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR:
>> (2554-28000) File
>> "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/sync.py", line
>> 114, in sync
>>
>> May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR:
>> (2554-28000) raise PulpExecutionException(_('Importer indicated a
>> failed response'))
>>
>> May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:ERROR:
>> (2554-28000) PulpExecutionException: Importer indicated a failed response
>>
>> May 05 13:33:06 pulp01.rap.local pulp[2554]: celery.worker.job:INFO: Task
>> pulp.server.async.tasks._release_resource[e8f32211-ccc5-4918-b4d5-ada23e15ecf4]
>> succeeded in 0.010533269s: None
>>
>>
>>
>> is there a clean way to fix this issue without Deleting the entire repo
>> and going through the process of setting this up again?
>>
>>
>>
>> Thanks for your time.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20150518/a29ba69d/attachment.htm>
More information about the Pulp-list
mailing list