[Pulp-list] Grinder orphanes packages after download
Willem Bos
bos.h.willem at gmail.com
Mon Nov 21 18:16:47 UTC 2011
Hi all,
I'm trying to sync a HP hosted repository (downloads.linux.hp.com).
Most of the packages are processed fine but a few are constantly
orphaned by Grinder after being downloaded. A few of them do show up
in /var/lib/pulp/packages/ but are not linked to a repo… I tried
creating the repo with '--feed=http://...' and also with an added
'--preserve_metada'.
Then I downloaded the packages with reposync, created metadata with
createrepo and placed this directory on a internal web server. With
the same result as mentioned above.
Finally, I tried '--feed=file:///...' (essentially bypassing Grinder)
and all packages were assigned correctly.
See below for detailed actions.
Any suggestions?
Regards,
Willem.
## FIRST ATTEMPT: ##
# pulp-admin repo create \
--id=hp-test \
--preserve_metadata \
--feed=http://downloads.linux.hp.com/SDR/downloads/proliantsupportpack/redhat/6/packages/x86_64/
Successfully created repository [ hp-test ]
# pulp-admin repo sync --id=hp-test --threads=1 --foreground
Sync for repository hp-test started
Sync: Finished
46/46 new items downloaded
0/46 existing items processed
Item Details:
RPMs: 46/46
# pulp-admin package search -f "^hp-.*"
+------------------------------------------+
Package Information
+------------------------------------------+
Name EVRA Filename
Repositories
hp-tg3 0:3.116j-3.src hp-tg3-3.116j-3.src.rpm
hp-test
hp-smh-templates 0:8.7.0.10-10.noarch
hp-smh-templates-8.7.0.10-10.noarch.rpm hp-test
hp-smh-templates 0:8.6.0.14-16.noarch
hp-smh-templates-8.6.0.14-16.noarch.rpm hp-test
...
# pulp-admin package search -f "^hp-snmp.*"
error: No packages found.
# ls -1d /var/lib/pulp/packages/hp-snmp*
/var/lib/pulp/packages/hp-snmp-agents
# cat /var/log/pulp/grinder.log
...
2011-11-20 21:13:35,155 13269:139726048700160: grinder.YumInfo:INFO:
activeobject:160 46 packages have been marked to be fetched
...
2011-11-20 21:13:37,708 13269:139726048700160:
grinder.ParallelFetch:INFO: ParallelFetch:177 46 items are marked to
be fetched
...
2011-11-20 21:14:53,340 13269:139725956445952: grinder.BaseFetch:INFO:
activeobject:160 Symlink missing in repo directory. Creating link
/var/lib/pulp//repos/SDR/downloads/proliantsupportpack/redhat/6/packages/x86_64//hp-snmp-agents-8.6.2.3-9.x86_64.rpm
to ../../../../../../../../packages/hp-snmp-agents/8.6.2.3/9/x86_64/e96/hp-snmp-agents-8.6.2.3-9.x86_64.rpm
...
2011-11-20 21:16:07,138 13269:139726048700160:
grinder.ParallelFetch:INFO: ParallelFetch:238 ParallelFetch: 46 items
successfully processed, 33 downloaded, 0 items had errors
...
2011-11-20 21:16:07,140 13269:139726048700160: grinder.RepoFetch:INFO:
RepoFetch:154 Cleaning any orphaned packages..
...
2011-11-20 21:16:07,143 13269:139726048700160: grinder.RepoFetch:INFO:
RepoFetch:190 Removing orphan package:
/var/lib/pulp//repos/SDR/downloads/proliantsupportpack/redhat/6/packages/x86_64/hp-snmp-agents-8.6.2.3-9.x86_64.rpm
...
2011-11-20 21:16:07,144 13269:139726048700160: grinder.RepoFetch:INFO:
RepoFetch:159 Processed <46> items in [154] seconds. Report: 46
successes, 33 downloads, 0 errors
2011-11-20 21:16:
...
# pulp-admin repo delete --id=hp-test
Successful deleted repository [ hp-test ]
# rm -r /var/lib/pulp/packages/{cpqacuxe,hpacucli,hpvca,hpdiags,hp-health,hp-snmp-agents}
## SECOND ATTEMPT: ##
# cat > /etc/yum.repos.d/hp-test.repo << EOF
[hp-test]
name=hp-test
baseurl=http://downloads.linux.hp.com/SDR/downloads/proliantsupportpack/redhat/6/packages/i686/
enabled=1
gpgcheck=0
EOF
# cd /var/www/html
# reposync --repoid=hp-test .
...
[hp-test: 11 of 31 ] Downloading hp-smh-templates-8.6.0.14-16.noarch.rpm
hp-smh-templates-8.6.0.14-16.noarch.rpm | 3.4
MB 00:02
[hp-test: 12 of 31 ] Downloading hp-snmp-agents-8.6.2.3-8.rhel6.i386.rpm
hp-snmp-agents-8.6.2.3-8.rhel6.i386.rpm | 1.7
MB 00:00
[hp-test: 13 of 31 ] Downloading hp-snmp-agents-8.7.0.23-16.rhel6.i386.rpm
hp-snmp-agents-8.7.0.23-16.rhel6.i386.rpm | 1.7
MB 00:00
...
# cd hp-test
# createrepo .
Spawning worker 0 with 31 pkgs
...
# zcat repodata/*-primary.xml.gz | grep "<name>hp-snmp"
<name>hp-snmp-agents</name>
<name>hp-snmp-agents</name>
# pulp-admin repo create \
--id=hp-test \
--feed=http://n004.localdomain/hp-test/
# pulp-admin repo sync --id=hp-test --threads=1 --foreground
# pulp-admin package search -f "^hp-snmp.*"
<output identical to first attempt>
# cat /var/log/pulp/grinder.log
<output identical to first attempt>
# ls -1d /var/lib/pulp/packages/hp-*
<output identical to first attempt>
# pulp-admin repo delete --id=hp-test
Successful deleted repository [ hp-test ]
# rm -r /var/lib/pulp/packages/{cpqacuxe,hpacucli,hpdiags,hp-health,hp-snmp-agents}
## THIRD ATTEMPT: ##
# pulp-admin repo create \
--id=hp-test \
--feed=file:///var/www/html/hp-test/
# pulp-admin repo sync --id=hp-test --threads=1 --foreground
pulp-admin repo sync --id=hp-test --threads=1 --foreground
Sync for repository hp-test started
Sync: Finished
31/31 new items downloaded
0/31 existing items processed
Item Details:
RPMs: 31/31
# pulp-admin package search -f "^hp-.*"
+------------------------------------------+
Package Information
+------------------------------------------+
Name EVRA Filename
Repositories
hp-snmp-agents 0:8.7.0.23-16.i386
hp-snmp-agents-8.7.0.23-16.rhel6.i386.rpm hp-test
hp-snmp-agents 0:8.6.2.3-8.i386
hp-snmp-agents-8.6.2.3-8.rhel6.i386.rpm hp-test
...
More information about the Pulp-list
mailing list