<div dir="ltr"><div><div>Hi Will,<br><br></div>It looks like the metadata you have is corrupted somehow and doesn't parse properly by the client software. It could be an error in liblxml, but it also could be an error in Pulp. Note that the difference in the number of repodata files is normal because those are older versions of the repository, but Pulp only is keeping 1 version.</div><div><br></div><div>To resolve this issue I recommend analyzing the metadata that is produced to understand exactly how it is malformed. Compare it to the metadata that you've mirror down to look for differences. Once you know how the metadata is malformed, I think you'll have an idea of how to patch Pulp to do the right thing.</div><div><br></div><div>Reproducing the issue against the developer install (Vagrant) would also be a good next step.</div><div><br></div><div>Feel free to send more questions or info.</div><div><br></div><div>-Brian</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 18, 2018 at 4:12 PM, Will Darton <span dir="ltr"><<a href="mailto:willdarton@gmail.com" target="_blank">willdarton@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div>I've been working on getting IBM's "new" yum repositories synchronized into pulp (ala foreman/katello).<br></div>After much teeth gnashing and keyboard banging we've gotten IBM's yum and rpm packages installed and functioning. <br><br></div>If I mirror IBM's repositories locally and point at them, yum has no issue with them<br><br></div>However if I sync them into pulp and distribute them that way and update the yum client to point to it I get the folllowing:<br><br><font size="2" face="sans-serif"># yum
repolist
</font>
<br><font size="2" face="sans-serif">AIX_Toolbox_noarch/primary
| 26 kB <span class="m_3105820301482695687gmail-aBn"><span class="m_3105820301482695687gmail-aQJ">00:00:00</span></span>
</font>
<br><font size="2" face="sans-serif">Traceback (most recent call last):</font>
<br><font size="2" face="sans-serif"> File "/usr/bin/yum",
line 29, in <module></font>
<br><font size="2" face="sans-serif"> yummain.user_main(sys.argv[1:]<wbr>,
exit_code=True)</font>
<br><font size="2" face="sans-serif"> File "/opt/freeware/share/yum-cli/y<wbr>ummain.py",
line 288, in user_main</font>
<br><font size="2" face="sans-serif"> errcode = main(args)</font>
<br><font size="2" face="sans-serif"> File "/opt/freeware/share/yum-cli/y<wbr>ummain.py",
line 140, in main</font>
<br><font size="2" face="sans-serif"> result, resultmsgs = base.doCommands()</font>
<br><font size="2" face="sans-serif"> File "/opt/freeware/share/yum-cli/c<wbr>li.py",
line 440, in doCommands</font>
<br><font size="2" face="sans-serif"> return self.yum_cli_commands[self.bas<wbr>ecmd].doCommand(self,
self.basecmd, self.extcmds)</font>
<br><font size="2" face="sans-serif"> File "/opt/freeware/share/yum-cli/y<wbr>umcommands.py",
line 864, in doCommand</font>
<br><font size="2" face="sans-serif"> base.repos.populateSack()</font>
<br><font size="2" face="sans-serif"> File "/opt/freeware/lib/python2.7/s<wbr>ite-packages/yum/repos.py",
line 294, in populateSack</font>
<br><font size="2" face="sans-serif"> sack.populate(repo, mdtype,
callback, cacheonly)</font>
<br><font size="2" face="sans-serif"> File "/opt/freeware/lib/python2.7/s<wbr>ite-packages/yum/yumRepo.py",
line 186, in populate</font>
<br><font size="2" face="sans-serif"> dobj = repo_cache_function(xml,
csum)</font>
<br><font size="2" face="sans-serif"> File "/opt/freeware/lib/python2.7/s<wbr>ite-packages/sqlitecachec.py",
line 46, in getPrimary</font>
<br><font size="2" face="sans-serif"> self.repoid))</font>
<br><font size="2" face="sans-serif">TypeError: Parsing primary.xml error:
Start tag expected, '<' not found</font>
<br>
<div><div><div><br></div><div><br></div><div>I recall from ages ago that this was an issue with the version of libxml on the RHEL client. In this case these are IBM supplied rpms for the ppc architecture so its very difficult to find "newer" versions of them</div><div><br></div><div>Here's some package information as well if its helpful</div><div><br></div><div><font size="2" face="sans-serif"># yum --version</font>
<br><font size="2" face="sans-serif">3.4.3</font>
<br><font size="2" face="sans-serif"> Installed: yum-3.4.3-5.noarch
at 2018-01-18 15:50</font>
<br><font size="2" face="sans-serif"> Built : None at
2017-08-03 07:56</font>
<br><font size="2" face="sans-serif"> Committed: Sangamesh Mallayya
<<a href="mailto:sangamesh.swamy@in.ibm.com" target="_blank">sangamesh.swamy@in.ibm.com</a>> at 2017-08-04</font>
<br>
<br><font size="2" face="sans-serif">AIX-rpm-7.2.2.0-8.ppc</font>
<br><font size="2" face="sans-serif">bash-4.2-9.ppc</font>
<br><font size="2" face="sans-serif">bzip2-1.0.6-2.ppc</font>
<br><font size="2" face="sans-serif">ca-certificates-2016.10.7-2.pp<wbr>c</font>
<br><font size="2" face="sans-serif">curl-7.52.1-1.ppc</font>
<br><font size="2" face="sans-serif">db-4.8.24-4.ppc</font>
<br><font size="2" face="sans-serif">gdbm-1.8.3-5.ppc</font>
<br><font size="2" face="sans-serif">gettext-0.17-8.ppc</font>
<br><font size="2" face="sans-serif">glib2-2.14.6-2.ppc</font>
<br><font size="2" face="sans-serif">info-5.0-2.ppc</font>
<br><font size="2" face="sans-serif">pysqlite-1.1.7-2.ppc</font>
<br><font size="2" face="sans-serif">python-2.7.10-1.ppc</font>
<br><font size="2" face="sans-serif">python-devel-2.7.10-1.ppc</font>
<br><font size="2" face="sans-serif">python-iniparse-0.4-1.noarch</font>
<br><font size="2" face="sans-serif">python-pycurl-7.19.3-1.ppc</font>
<br><font size="2" face="sans-serif">python-tools-2.7.10-1.ppc</font>
<br><font size="2" face="sans-serif">python-urlgrabber-3.10.1-1.noa<wbr>rch</font>
<br><font size="2" face="sans-serif">readline-6.2-3.ppc</font>
<br><font size="2" face="sans-serif">sqlite-3.15.2-1.ppc</font>
<br><font size="2" face="sans-serif">yum-3.4.3-5.noarch</font>
<br><font size="2" face="sans-serif">yum-metadata-parser-1.1.4-2.pp<wbr>c</font>
<br><font size="2" face="sans-serif">zlib-1.2.5-6.ppc</font></div><div><br></div><div><br></div><div>And here are the formats of the repodata directories respectively</div><div><br></div><div>pulp version:</div><div><font size="2" face="sans-serif"># ll ppc/repodata/</font>
<br><font size="2" face="sans-serif">total 1008</font>
<br><font size="2" face="sans-serif">-rw-r--r--. 1 apache apache 505877 Jan
17 11:59 04eb6a8e3b8fef27219496b6c01ef7<wbr>5a9135854699b7c45c8e13a2ea4ef8<wbr>0c16-filelists.xml.gz</font>
<br><font size="2" face="sans-serif">-rw-r--r--. 1 apache apache 153180 Jan
17 11:59 3b892d0959cac716766bdc1d7c7574<wbr>7f4d1c7f1364d54468fd3b6ac2a2ba<wbr>9b6c-primary.xml.gz</font>
<br><font size="2" face="sans-serif">-rw-r--r--. 1 apache apache 124
Jan 17 11:59 a27718cc28ec6d71432e0ef3e6da54<wbr>4b7f9d93f6bb7d0a55aacd592d0314<wbr>4b70-comps.xml</font>
<br><font size="2" face="sans-serif">-rw-r--r--. 1 apache apache
92 Jan 17 11:59 e48fdda4b1587f90b7f04d1e6326d2<wbr>60d7fa587a520bcd6fa74e94d0177c<wbr>bb41-updateinfo.xml.gz</font>
<br><font size="2" face="sans-serif">-rw-r--r--. 1 apache apache 352329 Jan
17 11:59 e7fdb522aaaa3693ad49fbb47eec76<wbr>b0336d1346cd2a4ccf405dff6b7276<wbr>d4bd-other.xml.gz</font>
<br><font size="2" face="sans-serif">-rw-r--r--. 1 apache apache 2162
Jan 17 11:59 repomd.xml</font>
<br>
<br>
</div><div>IBM repo mirrored locally.<br><font size="2" face="sans-serif"># ll /var/www/html/pub/ppc-7.2/repo<wbr>data/</font>
<br><font size="2" face="sans-serif">total 112</font>
<br><font size="2" face="sans-serif">-rw-r--r--. 1 root root 2064 Jul
27 10:02 0410861cec9162d680a30d85448717<wbr>eae4c5b64342d76c78235126b00159<wbr>b8de-other.xml.gz</font>
<br><font size="2" face="sans-serif">-rw-r--r--. 1 root root 7167 Jul
27 10:02 155714178fd87c7c14dccd26413a2b<wbr>15486c192faf3c9fec7edcc8f92215<wbr>14c8-filelists.xml.gz</font>
<br><font size="2" face="sans-serif">-rw-r--r--. 1 root root 7726 Jul
27 10:02 5c607a99d19877b8326f1cabb23b97<wbr>607a7744625af75597ea6c1eb64dba<wbr>d736-filelists.xml.gz</font>
<br><font size="2" face="sans-serif">-rw-r--r--. 1 root root 8113 Jul
27 10:02 6f86da92a38e4bc2a350450755134b<wbr>ef82c3c02ca56558bdd8f11862b959<wbr>51df-primary.sqlite.bz2</font>
<br><font size="2" face="sans-serif">-rw-r--r--. 1 root root 6286 Jul
27 10:02 73de6af5232700a5e5c8f5d26be562<wbr>4de959b3604700b75d1d8555769780<wbr>e45d-other.sqlite.bz2</font>
<br><font size="2" face="sans-serif">-rw-r--r--. 1 root root 2373 Jul
27 10:02 7de71092ff85e00b5298868c9800ab<wbr>382fe9811fbb604b38f29a2ed093db<wbr>c2fe-other.xml.gz</font>
<br><font size="2" face="sans-serif">-rw-r--r--. 1 root root 8221 Jul
27 10:02 91f305f8049c4515e8edb74aeabc8c<wbr>83f208744912013f7b5dc28586aede<wbr>d236-primary.sqlite.bz2</font>
<br><font size="2" face="sans-serif">-rw-r--r--. 1 root root 10817 Jul 27
10:02 96600a102ad1904fadf3005c8e6ca6<wbr>c418d4ccde92132f48cbb22a68891d<wbr>4d47-filelists.sqlite.bz2</font>
<br><font size="2" face="sans-serif">-rw-r--r--. 1 root root 3832 Jul
27 10:02 9d2f104b8df5e04cb901daf712e21d<wbr>547df7cebc383eca3a4c757afa0370<wbr>8df3-primary.xml.gz</font>
<br><font size="2" face="sans-serif">-rw-r--r--. 1 root root 11292 Jul 27
10:02 a65f0a6a16c940f5afd48e307a9bae<wbr>00ec0285d52dfb3be5408ef95d88d6<wbr>cf8b-filelists.sqlite.bz2</font>
<br><font size="2" face="sans-serif">-rw-r--r--. 1 root root 7070 Jul
27 10:02 c167a1149e6e47b75b3d62897e84d2<wbr>95501a51124f2db61feb029c920215<wbr>e2d6-other.sqlite.bz2</font>
<br><font size="2" face="sans-serif">-rw-r--r--. 1 root root 19719 Jul 27
10:02 cbcdd38f63d13f02324609a43b3c09<wbr>8a56b940d0189ab0157474f715eef0<wbr>b968-primary.xml</font>
<br><font size="2" face="sans-serif">-rw-r--r--. 1 root root 2977 Jul
27 10:02 repomd.xml<br></font></div><div><br></div><div><br><font size="2" face="sans-serif"></font></div><div>I doubt anyone has much time with IBM's yum and even less likely that anyone is using pulp to manage the repos, but thought I would give it a whirl..</div><div><br></div><div>Anyone have any ideas which direction to go here?</div><div><br></div><div><br></div><div><br></div><div><br clear="all"><div><div><div class="m_3105820301482695687gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr">Will Darton<br><br></div><div dir="ltr"><span style="color:rgb(255,0,0);font-family:"arial black",sans-serif">RHC{A,DS,E,VA,SA} </span><font face="arial black, sans-serif" color="#ff0000">130-047-673</font><br><span style="font-size:small;font-family:"arial black",sans-serif;color:rgb(255,0,0)"><br></span><font style="font-size:small">“<span style="color:rgb(51,51,51);font-family:Verdana,Arial,sans-serif;font-size:13px;line-height:18.2px;background-color:rgb(246,246,245)">There is excellence all around you. You need only to be aware to stop and savor it.</span>” - Anton Ego</font><span style="font-size:small"> </span><span style="font-family:arial black,sans-serif;color:rgb(255,0,0)"><br></span> </div></div></div></div></div></div></div></div></div></div></div></div></div>
</div></div></div></div></div>
<br>______________________________<wbr>_________________<br>
Pulp-dev mailing list<br>
<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a><br>
<br></blockquote></div><br></div>