[Pulp-list] Key-error on repo sync of distribution files of RHEL 5 Workstation/Client
Ditchfield, Harold (Calgary)
Harold.Ditchfield at nexencnoocltd.com
Thu Mar 13 16:48:16 UTC 2014
OK
I think I may have found the issue with RHEL 5 Workstation sync failing on distribution files
I compared my RHEL 5 Workstation repo (which was failing) against a RHEL 5 Server repo (which was working). Stepping through syncs on both sides, on RHEL 5 Workstation sync, the following errors were recorded in /var/log/pulp/nectar.log (in addition to the original error in ~/.pulp/admin.log recordered earlier)
...
2014-03-13 08:29:49,355 nectar.downloaders.threaded:ERROR: Download of https://cdn.redhat.com/content/dist/rhel/workstation/5/5Client/x86_64/os/images/pxeboot/initrd.img failed with code 404: Not Found
2014-03-13 08:29:49,392 nectar.downloaders.threaded:ERROR: Download of https://cdn.redhat.com/content/dist/rhel/workstation/5/5Client/x86_64/os/images/boot.iso failed with code 404: Not Found
2014-03-13 08:29:49,476 nectar.downloaders.threaded:ERROR: Download of https://cdn.redhat.com/content/dist/rhel/workstation/5/5Client/x86_64/os/images/pxeboot/vmlinuz failed with code 404: Not Found
2014-03-13 08:29:49,620 nectar.downloaders.threaded:ERROR: Download of https://cdn.redhat.com/content/dist/rhel/workstation/5/5Client/x86_64/os/images/xen/vmlinuz failed with code 404: Not Found
2014-03-13 08:29:49,628 nectar.downloaders.threaded:ERROR: Download of https://cdn.redhat.com/content/dist/rhel/workstation/5/5Client/x86_64/os/images/minstg2.img failed with code 404: Not Found
2014-03-13 08:29:49,794 nectar.downloaders.threaded:ERROR: Download of https://cdn.redhat.com/content/dist/rhel/workstation/5/5Client/x86_64/os/images/diskboot.img failed with code 404: Not Found
2014-03-13 08:29:49,876 nectar.downloaders.threaded:ERROR: Download of https://cdn.redhat.com/content/dist/rhel/workstation/5/5Client/x86_64/os/images/xen/initrd.img failed with code 404: Not Found
2014-03-13 08:29:49,940 nectar.downloaders.threaded:ERROR: Download of https://cdn.redhat.com/content/dist/rhel/workstation/5/5Client/x86_64/os/images/stage2.img failed with code 404: Not Found
...
These errors were not duplicated when I do the RHEL 5 Server sync.
I compared both the contents of the respective treeinfos (which were similar) but the images directory for RHEL 5 Workstation is empty even the the treeinfo says files are there. I believe this is where the sync is failing.
To get around this, I am just adding --skip=distribution to my RHEL 5 Workstation repo and everything flows nicely
Thanks
Harold
#####
I have added snippets of both the treeinfos and the contents of the images directory on both repos
#
### RHEL 5 Workstation - treeinfo
#
# curl -v https://cdn.redhat.com/content/dist/rhel/workstation/5/5Client/x86_64/os/treeinfo --cacert <CACERT> --cert <CERT> --key <KEY>
...
[images-xen]
initrd = images/xen/initrd.img
kernel = images/xen/vmlinuz
[stage2]
instimage = images/minstg2.img
mainimage = images/stage2.img
[images-x86_64]
client_productid.gz = Client/repodata/productid.gz
kernel = images/pxeboot/vmlinuz
client_comps-rhel5-client-core.xml = Client/repodata/bf6f9230de34cf661d5b32b7b400aa27a44ecc4c-comps-rhel5-client-core.xml
client_filelists.xml.gz = Client/repodata/6b48c1588e50490c95581da6a00aacac1fcc1947-filelists.xml.gz
client_primary.xml.gz = Client/repodata/83a2daa416ff1b576656fcde9942a878feceb8ae-primary.xml.gz
client_other.xml.gz = Client/repodata/fe53175fa95170f8a5f61c7da1eecd84855459d4-other.xml.gz
boot.iso = images/boot.iso
initrd = images/pxeboot/initrd.img
client_filelists.sqlite.bz2 = Client/repodata/03cad7c9ab0d0a03edecdc277f6c4a2256c438c5-filelists.sqlite.bz2
client_primary.sqlite.bz2 = Client/repodata/973aca31d29bf53d5f8991de9928d0365d7177b8-primary.sqlite.bz2
client_comps-rhel5-client-core.xml.gz = Client/repodata/f3313764d343443ee3a667cc8c1ad47b6e4e96a5-comps-rhel5-client-core.xml.gz
client_repomd = Client/repodata/repomd.xml
client_other.sqlite.bz2 = Client/repodata/f9f8226b553bdff322a44364b5cc2434bca2ca1b-other.sqlite.bz2
diskboot.img = images/diskboot.img
...
#
### RHEL 5 Server - treeinfo
#
# curl -v https://cdn.redhat.com/content/dist/rhel/server/5/5Server/x86_64/os/treeinfo -l --cacert <CACERT> --cert <CERT> --key <KEY>
...
[images-x86_64]
kernel = images/pxeboot/vmlinuz
server_filelists.xml.gz = Server/repodata/c39cb72d4c91f421065e7148981f8e32bcdae38f-filelists.xml.gz
server_primary.xml.gz = Server/repodata/a50108b5e810bc84474d5a423a096d536c8ecd9c-primary.xml.gz
server_other.sqlite.bz2 = Server/repodata/c534d188e424cae8a3c53159df297046b2b3d0a9-other.sqlite.bz2
server_productid.gz = Server/repodata/productid.gz
server_comps-rhel5-server-core.xml = Server/repodata/c4e4a37f4f4a2c78e2cf2fd1b8e2feb79e46bdc0-comps-rhel5-server-core.xml
server_updateinfo.xml.gz = Server/repodata/329cf55bc21b6c16fcc54fbcb0edd91bcf27d3bd-updateinfo.xml.gz
server_primary.sqlite.bz2 = Server/repodata/9fe5f344917a5b58321f4294e51f4e53adb8d367-primary.sqlite.bz2
server_comps-rhel5-server-core.xml.gz = Server/repodata/8badf71c14fd63da2ab307f1bc51e50caa83a2a4-comps-rhel5-server-core.xml.gz
boot.iso = images/boot.iso
initrd = images/pxeboot/initrd.img
server_repomd = Server/repodata/repomd.xml
diskboot.img = images/diskboot.img
server_filelists.sqlite.bz2 = Server/repodata/9d18189201e2869fedccd4f15c27a92607efcd3a-filelists.sqlite.bz2
server_other.xml.gz = Server/repodata/080fca7752409b03114f2efa43b6e27c14c8d1c5-other.xml.gz
[images-xen]
initrd = images/xen/initrd.img
kernel = images/xen/vmlinuz
[stage2]
instimage = images/minstg2.img
mainimage = images/stage2.img
...
#
### RHEL 5 Workstation - list of the images directory
#
# curl -v https://cdn.redhat.com/content/dist/rhel/workstation/5/5Client/x86_64/os/images -l --cacert <CACERT> --cert <CERT> --key <KEY>
...
<H1>Index of /98675/rcm/content/dist/rhel/workstation/5/5Client/x86_64/os/images</H1>
<PRE> <A HREF="?N=D">Name</A> <A HREF="?M=A">Last modified</A> <A HREF="?S=A">Size</A>
<HR>
<IMG SRC="/icons/dir.gif" ALT="[DIR]"> <A HREF="images/..">Parent Directory</A> 11-Mar-2014 16:58 1k
</PRE><HR>
</BODY></HTML>
...
#
### RHEL 5 Server - list of the images directory
#
# curl -v https://cdn.redhat.com/content/dist/rhel/server/5/5Server/x86_64/os/images -l --cacert <CACERT> --cert <CERT> --key <KEY>
...
<H1>Index of /98675/rcm/content/dist/rhel/server/5/5Server/x86_64/os/images</H1>
<PRE> <A HREF="?N=D">Name</A> <A HREF="?M=A">Last modified</A> <A HREF="?S=A">Size</A>
<HR>
<IMG SRC="/icons/dir.gif" ALT="[DIR]"> <A HREF="images/..">Parent Directory</A> 12-Mar-2014 21:53 1k
<IMG SRC="/icons/generic.gif" ALT="[FILE]"> <A HREF="images/README">README</A> 10-Sep-2013 19:42 1k
<IMG SRC="/icons/generic.gif" ALT="[FILE]"> <A HREF="images/boot.iso">boot.iso</A> 10-Sep-2013 19:43 14.6M
<IMG SRC="/icons/generic.gif" ALT="[FILE]"> <A HREF="images/diskboot.img">diskboot.img</A> 10-Sep-2013 19:43 15.0M
<IMG SRC="/icons/generic.gif" ALT="[FILE]"> <A HREF="images/minstg2.img">minstg2.img</A> 10-Sep-2013 19:44 42.5M
<IMG SRC="/icons/dir.gif" ALT="[DIR]"> <A HREF="images/pxeboot/">pxeboot/</A> 19-Jan-2014 20:52 1k
<IMG SRC="/icons/generic.gif" ALT="[FILE]"> <A HREF="images/stage2.img">stage2.img</A> 10-Sep-2013 19:45 90.0M
<IMG SRC="/icons/dir.gif" ALT="[DIR]"> <A HREF="images/xen/">xen/</A> 19-Jan-2014 20:34 1k
</PRE><HR>
</BODY></HTML>
...
________________________________
From: Ditchfield, Harold (Calgary)
Sent: Monday, March 10, 2014 5:52 AM
To: Christina Plummer
Cc: pulp-list at redhat.com
Subject: RE: [Pulp-list] Key-error on repo sync of distribution files of RHEL 5 Workstation/Client
Hey Christina
Unfortunately I haven't found the solution yet .. I hope to be able to devote some time this week to step through the download process so I can identify what the issue is ... I also have an alternate server (RedHat 6) to test on as well. I'll let the list know how it works out
Thanks
Harold
________________________________
From: Christina Plummer [cplummer at gmail.com]
Sent: Friday, March 07, 2014 1:29 PM
To: Ditchfield, Harold (Calgary)
Cc: pulp-list at redhat.com
Subject: Re: [Pulp-list] Key-error on repo sync of distribution files of RHEL 5 Workstation/Client
Hi Harold,
Did you ever find a solution to this one? I am in the process of migrating to a new Pulp 2.3.1 server and have been running into the same issue with the RHEL5 repos. I am getting the error on the Distributions even when I use my old Pulp 2.1.3 server as the feed instead of the Red Hat CDN.
Thanks,
Christina
On Tue, Jan 14, 2014 at 3:31 PM, Ditchfield, Harold (Calgary) <Harold.Ditchfield at nexencnoocltd.com<mailto:Harold.Ditchfield at nexencnoocltd.com>> wrote:
Has anybody seen the following and can point me in the right direction ..
Receive the following error when I try to sync the RHEL 5 Workstation .. packages (repository content downloads fine) but not the distribution files.
####### pulp-admin rpm repo sync run --repo-id=rhel-5-workstation
...
Downloading distribution files...
[=============================================== ] 94%
Distributions: 17/18 items
Errors encountered during distribution sync:
An unexpected error has occurred. More information can be found in the client
log file ~/.pulp/admin.log.
###### This is the corresponding error in ~/.pulp/admin.log
2014-01-13 13:32:18,720 - ERROR - Client-side exception occurred
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/pulp/client/extensions/core.py", line 478, in run
exit_code = Cli.run(self, args)
File "/usr/lib/python2.6/site-packages/okaara/cli.py", line 974, in run
exit_code = command_or_section.execute(self.prompt, remaining_args)
File "/usr/lib/python2.6/site-packages/pulp/client/extensions/extensions.py", line 224, in execute
return self.method(*arg_list, **clean_kwargs)
File "/usr/lib/python2.6/site-packages/pulp/client/commands/repo/sync_publish.py", line 101, in run
status.display_group_status(self.context, self.renderer, task_group_id)
File "/usr/lib/python2.6/site-packages/pulp/client/commands/repo/status/status.py", line 64, in display_group_status
_display_status(context, renderer, task_list)
File "/usr/lib/python2.6/site-packages/pulp/client/commands/repo/status/status.py", line 96, in _display_status
_display_task_status(context, renderer, task.task_id, quiet_waiting=quiet_waiting)
File "/usr/lib/python2.6/site-packages/pulp/client/commands/repo/status/status.py", line 123, in _display_task_status
renderer.display_report(response.response_body.progress)
File "/usr/lib/python2.6/site-packages/pulp_rpm/extension/admin/status.py", line 77, in display_report
self.render_distribution_sync_step(progress_report)
File "/usr/lib/python2.6/site-packages/pulp_rpm/extension/admin/status.py", line 160, in render_distribution_sync_step
'message' : error[1]['error_message'],
KeyError: 'error_message'
###### Pulp and OS version
Net new install of pulp-server-2.3.1-1.el6.noarch running on Oracle Linux 6.4 (so far everything else works including RHEL 5 Server repos)
###### pulp repo config
Id: rhel-5-workstation
Display Name: rhel-5-workstation
Description: None
Content Unit Counts:
Erratum: 1404
Package Group: 27
Rpm: 6550
Yum Repo Metadata File: 1
Notes:
Importers:
Config:
Feed: https://cdn.redhat.com/content/dist/rhel/workstation/5/5Client/x86_64/os
SSL CA Cert: <INSERT CERT HERE>
Id: yum_importer
Importer Type Id: yum_importer
Last Sync: 2014-01-14T10:05:18-07:00
Repo Id: rhel-5-workstation
Scheduled Syncs:
Distributors:
Auto Publish: True
Config:
Checksum Type: sha1
Http: False
Https: True
Relative URL: /content/dist/rhel/workstation/5/5Workstation/x86_64/os
Distributor Type Id: yum_distributor
Id: yum_distributor
Last Publish: 2014-01-14T10:06:24-07:00
Repo Id: rhel-5-workstation
Scheduled Publishes:
Auto Publish: False
Config:
Http: False
Https: True
Distributor Type Id: export_distributor
Id: export_distributor
Last Publish: None
Repo Id: rhel-5-workstation
Scheduled Publishes:
************************************************************
IMPORTANT NOTICE: This message is intended for the individual or entity to which it is addressed and may contain information that is privileged, confidential, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that copying, forwarding or other dissemination or distribution of this message is prohibited and that taking any action in reliance on the content of this message is to be avoided. Should you receive this e-mail in error, please notify the sender immediately via e-mail or call (403) 699-4000<tel:%28403%29%20699-4000> and delete this message from your system. Thank you.
************************************************************
_______________________________________________
Pulp-list mailing list
Pulp-list at redhat.com<mailto:Pulp-list at redhat.com>
https://www.redhat.com/mailman/listinfo/pulp-list
************************************************************
IMPORTANT NOTICE: This message is intended for the individual or entity to which it is addressed
and may contain information that is privileged, confidential, and/or exempt from disclosure under
applicable law. If you are not the intended recipient, you are hereby notified that copying,
forwarding or other dissemination or distribution of this message is prohibited and that taking
any action in reliance on the content of this message is to be avoided. Should you receive this
e-mail in error, please notify the sender immediately via e-mail or call (403) 699-4000
and delete this message from your system. Thank you.
************************************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20140313/98e08173/attachment.htm>
More information about the Pulp-list
mailing list