<div dir="ltr"><div>I continued the investigation and ended up filing a bug against Django[0]. I updated my initial patch to reflect the deeper understanding of the problem. The latest patch should generate fewer queries to the DB[1] during migration from Pulp 2 to 3. <br></div><div><br></div><div>[0] <a href="https://code.djangoproject.com/ticket/32089">https://code.djangoproject.com/ticket/32089</a></div><div><div>[1] <a href="https://github.com/pulp/pulpcore/pull/937">https://github.com/pulp/pulpcore/pull/937</a></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Sep 29, 2020 at 9:58 AM Daniel Alley <<a href="mailto:dalley@redhat.com">dalley@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">The createrepo_c issue is a packaging problem, our Python packages include the correct patch but the RPMs don't have the patch applied. We're going to get this fixed within the next day or two.<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Sep 28, 2020 at 3:00 AM Winberg Adam <<a href="mailto:Adam.Winberg@smhi.se" target="_blank">Adam.Winberg@smhi.se</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir="ltr"> <div id="gmail-m_-7634902459372607200gmail-m_7201400071377926734divtagdefaultwrapper" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif" dir="ltr"> <p>Thanks! That's a very nice patch, after trying for a week to get the migration to work and continously searching for and deleting duplicate pulp2 content, the migration worked right away after this patch. :)</p> <p><br> </p> <p>Or, at least regarding the duplicate content errors, however I still have problems with a createrepo_c error, same as described in <a href="https://pulp.plan.io/issues/7193" id="gmail-m_-7634902459372607200gmail-m_7201400071377926734LPlnk140001" target="_blank">https://pulp.plan.io/issues/7193</a> (huge input lookup). </p> <p><br> </p> <p>That error only arise for a few packages so that is manageable for me to purge from pulp2 before migration, but it would be nice to not have to do that..</p> <p><br> </p> <p>//Adam</p> <p><br> </p> <p><br> </p> <br> <br> <div style="color:rgb(0,0,0)"> <hr style="display:inline-block;width:98%"> <div id="gmail-m_-7634902459372607200gmail-m_7201400071377926734divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Dennis Kliban <<a href="mailto:dkliban@redhat.com" target="_blank">dkliban@redhat.com</a>><br> <b>Sent:</b> 27 September 2020 17:35<br> <b>To:</b> Winberg Adam<br> <b>Cc:</b> Ina Panova; <a href="mailto:pulp-list@redhat.com" target="_blank">pulp-list@redhat.com</a><br> <b>Subject:</b> Re: [Pulp-list] 2to3migration fails on 'Package' object has no attribute '_remote_artifact_saver_cas'</font> <div> </div> </div> <div> <div dir="ltr"> <div dir="ltr">Here is a patch for pulpcore that fixes this problem[0].<br> </div> <div><br> </div> <div>[0] <a href="https://github.com/pulp/pulpcore/pull/937" id="gmail-m_-7634902459372607200gmail-m_7201400071377926734LPlnk119806" target="_blank"> https://github.com/pulp/pulpcore/pull/937</a></div> <div><br> </div> <div class="gmail_quote"> <div dir="ltr" class="gmail_attr">On Mon, Sep 21, 2020 at 7:17 AM Winberg Adam <<a href="mailto:Adam.Winberg@smhi.se" target="_blank">Adam.Winberg@smhi.se</a>> wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir="ltr"> <div id="gmail-m_-7634902459372607200gmail-m_7201400071377926734gmail-m_9038961628723507811divtagdefaultwrapper" dir="ltr" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif"> <p>Thank you for your reply - yes I did clean orphans after i removed the pulp2 repos that i suspected might be the cause. But as you say, there might be some other cause for this in my case. </p> <p><br> </p> <p>At the moment I have adjusted the code so the iteration only runs if the '<span>_remote_artifact_saver_cas' attribute is present. Don't know if the result will be any good though, running the migration right now. </span></p> <p><span><br> </span></p> <p><span>//Adam</span></p> <br> <br> <div style="color:rgb(0,0,0)"> <hr style="display:inline-block;width:98%"> <div id="gmail-m_-7634902459372607200gmail-m_7201400071377926734gmail-m_9038961628723507811divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Ina Panova <<a href="mailto:ipanova@redhat.com" target="_blank">ipanova@redhat.com</a>><br> <b>Sent:</b> 21 September 2020 13:09<br> <b>To:</b> Winberg Adam<br> <b>Cc:</b> <a href="mailto:pulp-list@redhat.com" target="_blank">pulp-list@redhat.com</a><br> <b>Subject:</b> Re: [Pulp-list] 2to3migration fails on 'Package' object has no attribute '_remote_artifact_saver_cas'</font> <div> </div> </div> <div> <div dir="ltr"> <div>Hi,</div> <div>the provided steps in the mentioned issue are the steps to reproduce the issue, however, unfortunately, this does not necessarily mean that this is the root cause of the manifested problem.</div> <div>Apparently we need to find a fix to properly handle duplicated declarative content in a batch.</div> <div><br> </div> <div>Looking at the steps you have tried to bypass the issue, have you run orphan clean up after pulp2 repos removal? <br> </div> <div> <div> <div dir="ltr"> <div dir="ltr"> <div> <div dir="ltr"><br> <br> --------<br> Regards,<br> <br> Ina Panova<br> Senior Software Engineer| Pulp| Red Hat Inc.<br> <br> "Do not go where the path may lead,<br> go instead where there is no path and leave a trail."<br> </div> </div> </div> </div> </div> <br> </div> </div> <br> <div class="gmail_quote"> <div dir="ltr" class="gmail_attr">On Sun, Sep 20, 2020 at 9:05 AM Winberg Adam <<a href="mailto:Adam.Winberg@smhi.se" target="_blank">Adam.Winberg@smhi.se</a>> wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir="ltr"> <div id="gmail-m_-7634902459372607200gmail-m_7201400071377926734gmail-m_9038961628723507811gmail-m_-2658754671186061360divtagdefaultwrapper" dir="ltr" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif"> <p>Hi,</p> <p><br> </p> <p>When running the 2to3migration for the 'rpm' plugin, I get the following error:</p> <p><br> </p> <p><span>AttributeError: 'Package' object has no attribute '_remote_artifact_saver_cas'</span><br> </p> <p><span><br> </span></p> <p><span><br> </span></p> <p><span>This is the same as specified in <a href="https://pulp.plan.io/issues/7147" id="gmail-m_-7634902459372607200gmail-m_7201400071377926734gmail-m_9038961628723507811gmail-m_-2658754671186061360LPlnk644454" target="_blank">https://pulp.plan.io/issues/7147</a>, and I actually had a couple of repos in pulp2 which shared the same feed. I removed the redundant repos, flushed the pulp3 db and reran the 2to3-migration but still got stuck on the same error. </span></p> <p><span><br> </span></p> <p><span>Anyone got any pointers how to resolve this?</span></p> <p><span><br> </span></p> <p><span>//Adam</span></p> </div> </div> _______________________________________________<br> Pulp-list mailing list<br> <a href="mailto:Pulp-list@redhat.com" target="_blank">Pulp-list@redhat.com</a><br> <a href="https://www.redhat.com/mailman/listinfo/pulp-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a></blockquote> </div> </div> </div> </div> </div> _______________________________________________<br> Pulp-list mailing list<br> <a href="mailto:Pulp-list@redhat.com" target="_blank">Pulp-list@redhat.com</a><br> <a href="https://www.redhat.com/mailman/listinfo/pulp-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a></blockquote> </div> </div> </div> </div> </div> </div> _______________________________________________<br> Pulp-list mailing list<br> <a href="mailto:Pulp-list@redhat.com" target="_blank">Pulp-list@redhat.com</a><br> <a href="https://www.redhat.com/mailman/listinfo/pulp-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a></blockquote></div> </blockquote></div>