[Pulp-dev] IBM's yum "repository" vs a normal sane repository

Brian Bouterse bbouters at redhat.com
Thu Jan 18 22:21:42 UTC 2018 

Hi Will,

It looks like the metadata you have is corrupted somehow and doesn't parse
properly by the client software. It could be an error in liblxml, but it
also could be an error in Pulp. Note that the difference in the number of
repodata files is normal because those are older versions of the
repository, but Pulp only is keeping 1 version.

To resolve this issue I recommend analyzing the metadata that is produced
to understand exactly how it is malformed. Compare it to the metadata that
you've mirror down to look for differences. Once you know how the metadata
is malformed, I think you'll have an idea of how to patch Pulp to do the
right thing.

Reproducing the issue against the developer install (Vagrant) would also be
a good next step.

Feel free to send more questions or info.

-Brian


On Thu, Jan 18, 2018 at 4:12 PM, Will Darton <willdarton at gmail.com> wrote:

> I've been working on getting IBM's "new" yum repositories synchronized
> into pulp (ala foreman/katello).
> After much teeth gnashing and keyboard banging we've gotten IBM's yum and
> rpm packages installed and functioning.
>
> If I mirror IBM's repositories locally and point at them, yum has no issue
> with them
>
> However if I sync them into pulp and distribute them that way and update
> the yum client to point to it I get the folllowing:
>
> # yum repolist
>
> AIX_Toolbox_noarch/primary
>                                                             |  26 kB
> 00:00:00
> Traceback (most recent call last):
>   File "/usr/bin/yum", line 29, in <module>
>     yummain.user_main(sys.argv[1:], exit_code=True)
>   File "/opt/freeware/share/yum-cli/yummain.py", line 288, in user_main
>     errcode = main(args)
>   File "/opt/freeware/share/yum-cli/yummain.py", line 140, in main
>     result, resultmsgs = base.doCommands()
>   File "/opt/freeware/share/yum-cli/cli.py", line 440, in doCommands
>     return self.yum_cli_commands[self.basecmd].doCommand(self,
> self.basecmd, self.extcmds)
>   File "/opt/freeware/share/yum-cli/yumcommands.py", line 864, in
> doCommand
>     base.repos.populateSack()
>   File "/opt/freeware/lib/python2.7/site-packages/yum/repos.py", line
> 294, in populateSack
>     sack.populate(repo, mdtype, callback, cacheonly)
>   File "/opt/freeware/lib/python2.7/site-packages/yum/yumRepo.py", line
> 186, in populate
>     dobj = repo_cache_function(xml, csum)
>   File "/opt/freeware/lib/python2.7/site-packages/sqlitecachec.py", line
> 46, in getPrimary
>     self.repoid))
> TypeError: Parsing primary.xml error: Start tag expected, '<' not found
>
>
> I recall from ages ago that this was an issue with the version of libxml
> on the RHEL client.  In this case these are IBM supplied rpms for the ppc
> architecture so its very difficult to find "newer" versions of them
>
> Here's some package information as well if its helpful
>
> # yum --version
> 3.4.3
>   Installed: yum-3.4.3-5.noarch at 2018-01-18 15:50
>   Built    : None at 2017-08-03 07:56
>   Committed: Sangamesh Mallayya <sangamesh.swamy at in.ibm.com> at 2017-08-04
>
> AIX-rpm-7.2.2.0-8.ppc
> bash-4.2-9.ppc
> bzip2-1.0.6-2.ppc
> ca-certificates-2016.10.7-2.ppc
> curl-7.52.1-1.ppc
> db-4.8.24-4.ppc
> gdbm-1.8.3-5.ppc
> gettext-0.17-8.ppc
> glib2-2.14.6-2.ppc
> info-5.0-2.ppc
> pysqlite-1.1.7-2.ppc
> python-2.7.10-1.ppc
> python-devel-2.7.10-1.ppc
> python-iniparse-0.4-1.noarch
> python-pycurl-7.19.3-1.ppc
> python-tools-2.7.10-1.ppc
> python-urlgrabber-3.10.1-1.noarch
> readline-6.2-3.ppc
> sqlite-3.15.2-1.ppc
> yum-3.4.3-5.noarch
> yum-metadata-parser-1.1.4-2.ppc
> zlib-1.2.5-6.ppc
>
>
> And here are the formats of the repodata directories respectively
>
> pulp version:
> # ll ppc/repodata/
> total 1008
> -rw-r--r--. 1 apache apache 505877 Jan 17 11:59
> 04eb6a8e3b8fef27219496b6c01ef75a9135854699b7c45c8e13a2ea4ef8
> 0c16-filelists.xml.gz
> -rw-r--r--. 1 apache apache 153180 Jan 17 11:59
> 3b892d0959cac716766bdc1d7c75747f4d1c7f1364d54468fd3b6ac2a2ba
> 9b6c-primary.xml.gz
> -rw-r--r--. 1 apache apache    124 Jan 17 11:59
> a27718cc28ec6d71432e0ef3e6da544b7f9d93f6bb7d0a55aacd592d03144b70-comps.xml
> -rw-r--r--. 1 apache apache     92 Jan 17 11:59
> e48fdda4b1587f90b7f04d1e6326d260d7fa587a520bcd6fa74e94d0177c
> bb41-updateinfo.xml.gz
> -rw-r--r--. 1 apache apache 352329 Jan 17 11:59
> e7fdb522aaaa3693ad49fbb47eec76b0336d1346cd2a4ccf405dff6b7276
> d4bd-other.xml.gz
> -rw-r--r--. 1 apache apache   2162 Jan 17 11:59 repomd.xml
>
> IBM repo mirrored locally.
> # ll /var/www/html/pub/ppc-7.2/repodata/
> total 112
> -rw-r--r--. 1 root root  2064 Jul 27 10:02 0410861cec9162d680a30d85448717
> eae4c5b64342d76c78235126b00159b8de-other.xml.gz
> -rw-r--r--. 1 root root  7167 Jul 27 10:02 155714178fd87c7c14dccd26413a2b
> 15486c192faf3c9fec7edcc8f9221514c8-filelists.xml.gz
> -rw-r--r--. 1 root root  7726 Jul 27 10:02 5c607a99d19877b8326f1cabb23b97
> 607a7744625af75597ea6c1eb64dbad736-filelists.xml.gz
> -rw-r--r--. 1 root root  8113 Jul 27 10:02 6f86da92a38e4bc2a350450755134b
> ef82c3c02ca56558bdd8f11862b95951df-primary.sqlite.bz2
> -rw-r--r--. 1 root root  6286 Jul 27 10:02 73de6af5232700a5e5c8f5d26be562
> 4de959b3604700b75d1d8555769780e45d-other.sqlite.bz2
> -rw-r--r--. 1 root root  2373 Jul 27 10:02 7de71092ff85e00b5298868c9800ab
> 382fe9811fbb604b38f29a2ed093dbc2fe-other.xml.gz
> -rw-r--r--. 1 root root  8221 Jul 27 10:02 91f305f8049c4515e8edb74aeabc8c
> 83f208744912013f7b5dc28586aeded236-primary.sqlite.bz2
> -rw-r--r--. 1 root root 10817 Jul 27 10:02 96600a102ad1904fadf3005c8e6ca6
> c418d4ccde92132f48cbb22a68891d4d47-filelists.sqlite.bz2
> -rw-r--r--. 1 root root  3832 Jul 27 10:02 9d2f104b8df5e04cb901daf712e21d
> 547df7cebc383eca3a4c757afa03708df3-primary.xml.gz
> -rw-r--r--. 1 root root 11292 Jul 27 10:02 a65f0a6a16c940f5afd48e307a9bae
> 00ec0285d52dfb3be5408ef95d88d6cf8b-filelists.sqlite.bz2
> -rw-r--r--. 1 root root  7070 Jul 27 10:02 c167a1149e6e47b75b3d62897e84d2
> 95501a51124f2db61feb029c920215e2d6-other.sqlite.bz2
> -rw-r--r--. 1 root root 19719 Jul 27 10:02 cbcdd38f63d13f02324609a43b3c09
> 8a56b940d0189ab0157474f715eef0b968-primary.xml
> -rw-r--r--. 1 root root  2977 Jul 27 10:02 repomd.xml
>
>
> I doubt anyone has much time with IBM's yum and even less likely that
> anyone is using pulp to manage the repos, but thought I would give it a
> whirl..
>
> Anyone have any ideas which direction to go here?
>
>
>
>
> Will Darton
>
> RHC{A,DS,E,VA,SA} 130-047-673
>
> “There is excellence all around you. You need only to be aware to stop
> and savor it.” - Anton Ego
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20180118/efd4d215/attachment.htm>

More information about the Pulp-dev mailing list