<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> <style type="text/css" style="display:none;"></style> </head> <body dir="ltr"> <div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr"> <p><span>>Ok, tried that now, synced a total of 72 advisories in 3-4 copy operations. </span></p> <p><span>This was a bit unclear I realize, but I created new pulp3 repos for appstream and baseos and synced them with our redhat cdn remote as source. Then I _copied_ a total of 72 advisories from this new appstream repo to a new empty repo, while using the new baseos repo as a 'dependency source' paired with another new empty repo. </span><br> </p> <p><span><br> </span></p> <p><span>//Adam</span></p> <br> <br> <div style="color: rgb(0, 0, 0);"> <hr tabindex="-1" style="display:inline-block; width:98%"> <div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> pulp-list-bounces@redhat.com <pulp-list-bounces@redhat.com> on behalf of Winberg Adam <Adam.Winberg@smhi.se><br> <b>Sent:</b> 30 September 2020 09:06<br> <b>To:</b> Daniel Alley<br> <b>Cc:</b> pulp-list@redhat.com<br> <b>Subject:</b> Re: [Pulp-list] rpm plugin: multirepo copy of errata not working</font> <div> </div> </div> <div> <div id="divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Helvetica,sans-serif"> <p>We're glad to help, Pulp is extremely useful for us so if we can contribute we are happy to do so. I created an issue for this:</p> <p><a href="https://pulp.plan.io/issues/7625" class="OWAAutoLink" id="LPlnk512857" previewremoved="true">https://pulp.plan.io/issues/7625</a><br> </p> <p><br> </p> <p>> <span> If you sync the Pulp 3 repository using the migrated remote, what happens?</span></p> <p><span>Do you mean syncing the '<span>my-new-repo1' repo with the migrated appstream remote?</span></span></p> <div><br> </div> <div>> <span>Are the Pulp 2 repositories you're migrating modified significantly from when they were originally synced?</span></div> <div><span>Well, in the case of the appstream repo it has been synced against the Red Hat source every night, so the content has certainly been modified. Otherwise than that, no. </span></div> <div><br> </div> <div>> <span>If you make brand new Pulp 3 repositories and try the same copy operation, does the same weird copy behavior occur?</span></div> <div><span>Do you mean new pulp3 repos synced using the appstream and baseos remotes (w. content from redhat cdn) and then using that as the source of the copy? Ok, tried that now, synced a total of 72 advisories in 3-4 copy operations. The copy itself between my new pulp3 appstream repo and the new empty repo works well and 610 rpm packages has been copied. But there are no dependencies copied from the new baseos repo, no content at all. </span></div> <div><span><br> </span></div> <div><span>//Adam</span></div> <div><br> </div> <br> <br> <div style="color:rgb(0,0,0)"> <hr tabindex="-1" style="display:inline-block; width:98%"> <div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Daniel Alley <dalley@redhat.com><br> <b>Sent:</b> 29 September 2020 20:28<br> <b>To:</b> Winberg Adam<br> <b>Cc:</b> pulp-list@redhat.com<br> <b>Subject:</b> Re: [Pulp-list] rpm plugin: multirepo copy of errata not working</font> <div> </div> </div> <div> <div dir="ltr"> <div>Hi Adam,</div> <div><br> </div> <div>Thank you for providing feedback on some of these rough edges, it is <i>extremely</i> helpful.<br> </div> <div><br> </div> <div>There's a couple of things you can try that would give us some useful information. If you sync the Pulp 3 repository using the migrated remote, what happens? Does the repository change or does it stay the same?<br> </div> <div><br> </div> <div>Are the Pulp 2 repositories you're migrating modified significantly from when they were originally synced? If you make brand new Pulp 3 repositories and try the same copy operation, does the same weird copy behavior occur?<br> </div> <div><br> </div> <div>Also, if it wouldn't be too much trouble, could you file this as an issue on our bug tracker (<a href="https://pulp.plan.io/" id="LPlnk900158" previewremoved="true">https://pulp.plan.io/</a>) with the details? We can continue discussion here but if there is a lot of information it's better to keep it in one place on the issue.<br> </div> <div><br> </div> <div>Thanks again,</div> <div>Daniel<br> </div> <div><br> </div> </div> <br> <div class="gmail_quote"> <div dir="ltr" class="gmail_attr">On Tue, Sep 29, 2020 at 8:21 AM Winberg Adam <<a href="mailto:Adam.Winberg@smhi.se">Adam.Winberg@smhi.se</a>> wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex"> <div dir="ltr"> <div id="gmail-m_5620411293537284787divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif"> <p>I applied the patch from </p> <p><a href="https://github.com/pulp/pulp_rpm/commit/712abdf1abb95c969b54fd2968a573189b77bcba" id="gmail-m_5620411293537284787LPlnk312482" target="_blank">https://github.com/pulp/pulp_rpm/commit/712abdf1abb95c969b54fd2968a573189b77bcba</a><br> </p> <p><br> </p> <p>and the the copy then went through without errors. </p> <p><br> </p> <p>I'm a bit confused by the result however. I copied 16 advisories from the appstreams repo to my new empty repo and the copy ended up with copying all modulemds and almost all packages (11466 of 12053). That doesnt seem right to me. And my other new repo ('my-new-repo2') is still empty, meaning that of those 11000 packages there were none that had any dependencies from the baseos repo which also strikes me as odd. If i set 'dependency_solving=True' (contrary to the documentation) I end up with the same amount of packages in my 'my-new-repo1' and 2 packages in 'my-new-repo2'. </p> <p><br> </p> <p>Am I misunderstanding something about this functionality?</p> <p><br> </p> <p>//Adam</p> <p><br> </p> <p><br> </p> <br> <br> <div style="color:rgb(0,0,0)"> <hr style="display:inline-block; width:98%"> <div id="gmail-m_5620411293537284787divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Winberg Adam<br> <b>Sent:</b> 28 September 2020 16:44<br> <b>To:</b> <a href="mailto:pulp-list@redhat.com" target="_blank">pulp-list@redhat.com</a><br> <b>Subject:</b> rpm plugin: multirepo copy of errata not working</font> <div> </div> </div> <div> <div id="gmail-m_5620411293537284787divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif"> <p>I have succeeded in migrating my pulp2 content to pulp3 and all repos look complete (i've reran the migration and no new repoversions are generated). </p> <p><br> </p> <p>In an attempt to test the multirepo copy functionality described at </p> <p><a href="https://pulp-rpm.readthedocs.io/en/latest/workflows/copy.html#recipes" id="gmail-m_5620411293537284787LPlnk69478" target="_blank">https://pulp-rpm.readthedocs.io/en/latest/workflows/copy.html#recipes</a> </p> <p><br> </p> <p>, I created a couple of new, empty repos and tried to copy errata from my migrated RHEL8-appstream repo to them: </p> <p></p> <div><span style="font-family:Consolas,Courier,monospace; font-size:8pt">POST /pulp/api/v3/rpm/copy/</span></div> <div><span style="font-family:Consolas,Courier,monospace; font-size:8pt">config:=[</span></div> <div><span style="font-family:Consolas,Courier,monospace; font-size:8pt"> {"source_repo_version": "</span><span style="font-family:Consolas,Courier,monospace; font-size:8pt"><rhel8-appstream-version-href></span><span style="font-family:Consolas,Courier,monospace; font-size:8pt">", "dest_repo": "my-new-repo1</span><span style="font-family:Consolas,Courier,monospace; font-size:8pt">", "content": [</span><span style="font-family:Consolas,Courier,monospace; font-size:8pt">$ADVISORY_HREF1]},</span></div> <div><span style="font-family:Consolas,Courier,monospace; font-size:8pt"> {"source_repo_version": "</span><span style="font-family:Consolas,Courier,monospace; font-size:8pt"><rhel8-baseos-version-href></span><span style="font-family:Consolas,Courier,monospace; font-size:8pt">", "dest_repo": "my-new-repo2", "content": []},</span></div> <div><span style="font-family:Consolas,Courier,monospace; font-size:8pt">]</span></div> <div><span style="font-family:Consolas,Courier,monospace; font-size:8pt">dependency_solving=False</span></div> <br> <p></p> <p><br> </p> <p><span style="font-size:12pt">All looks correct, but the operation ultimately fails with the following error:</span><br> </p> <p></p> <div><span style="font-family:Consolas,Courier,monospace; font-size:8pt"> "description": "Modulemd matching query does not exist.",</span></div> <div><span style="font-family:Consolas,Courier,monospace; font-size:8pt"> "traceback": " File \"/usr/lib/python3.6/site-packages/rq/worker.py\", line 883, in perform_job\n rv = job.perform()\n File \"/usr/lib/python3.6/site-packages/rq/job.py\", line 657, in perform\n self._result = self._execute()\n File \"/usr/lib/python3.6/site-packages/rq/job.py\", line 663, in _execute\n return self.func(*self.args, **self.kwargs)\n File \"/usr/lib64/python3.6/contextlib.py\", line 52, in inner\n return func(*args, **kwds)\n File \"/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/copy.py\", line 167, in copy_content\n content_to_copy |= find_children_of_content(content_to_copy, source_repo_version)\n File \"/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/copy.py\", line 74, in find_children_of_content\n name=name, stream=stream, version=version, context=context, arch=arch)\n File \"/usr/lib/python3.6/site-packages/django/db/models/query.py\", line 408, in get\n self.model._meta.object_name\n"</span></div> <div><span style="font-family:Consolas,Courier,monospace; font-size:8pt"> },</span></div> <div><br> </div> Any ideas why this happens? <p></p> <p><br> </p> <p>Regards,</p> <p>Adam</p> <p><br> </p> <p><br> </p> <p><br> </p> </div> </div> </div> </div> </div> _______________________________________________<br> Pulp-list mailing list<br> <a href="mailto:Pulp-list@redhat.com" target="_blank">Pulp-list@redhat.com</a><br> <a href="https://www.redhat.com/mailman/listinfo/pulp-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a></blockquote> </div> </div> </div> </div> </div> </div> </div> </body> </html>