<div dir="ltr"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><span class="gmail-im">
</span><span>No, this is with 0.4.0, since 0.5.0 is quite new I haven't updated yet. Will do so and try again. </span><span class="gmail-im">
</span></div></blockquote><div><br></div><div>If it's 0.4.0, then this is actually not surprising after all. This was a bug that was fixed in 0.5.0</div><div><br></div><div><a href="https://pulp.plan.io/issues/7280">https://pulp.plan.io/issues/7280</a></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Oct 23, 2020 at 12:45 PM Winberg Adam <<a href="mailto:Adam.Winberg@smhi.se">Adam.Winberg@smhi.se</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div id="gmail-m_-1790087884150906635divtagdefaultwrapper" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif" dir="ltr">
<p>> <span>This is with the very latest version of the migration plugin (0.5.0)?</span></p>
<span>No, this is with 0.4.0, since 0.5.0 is quite new I haven't updated yet. Will do so and try again. </span>
<div><br>
</div>
<div>> <span>Is it actually equally long, or just not 'reasonably' faster? <span style="font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:16px">Is
it only the "migrating rpm content to Pulp 3 erratum" sub-item which reports doing a lot of work or are many of them doing so?</span></span></div>
<div>I would say that the runtime is very similar. And it looks like many if not all sub-items are repeating their work but since the 'pulp3 erratum' is the longest running that's the one that stands out. Here are some progress reports on a subsequent run which
should only resulted in a handful of new rpm's:</div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> {</span><br>
</div>
<div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "code": "migrating.rpm.content",</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "done": 32348,</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "message": "Migrating rpm content to Pulp 3 rpm",</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "state": "completed",</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "suffix": null,</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "total": 32348 </span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> },</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> {</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "code": "premigrating.content.general",</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "done": 89690,</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "message": "Pre-migrating Pulp 2 ERRATUM content (general info)",</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "state": "completed",</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "suffix": null,</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "total": 89690 </span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> },</span></div>
<div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> {</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "code": "premigrating.content.general",</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "done": 39,</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "message": "Pre-migrating Pulp 2 RPM content (general info)",</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "state": "completed",</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "suffix": null,</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "total": 43</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> },</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> {</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "code": "migrating.rpm.content",</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "done": 89690,</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "message": "Migrating rpm content to Pulp 3 erratum",</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "state": "completed",</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "suffix": null,</span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "total": 89690 </span></div>
<div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> },</span></div>
<div><br>
</div>
Should these numbers be 0 on a subsequent run if nothing has changed?</div>
<div><br>
</div>
<div>//Adam</div>
<br>
</div>
<div><br>
<div style="color:rgb(0,0,0)">
<hr style="display:inline-block;width:98%">
<div id="gmail-m_-1790087884150906635divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Daniel Alley <<a href="mailto:dalley@redhat.com" target="_blank">dalley@redhat.com</a>><br>
<b>Sent:</b> 23 October 2020 17:33<br>
<b>To:</b> Winberg Adam<br>
<b>Cc:</b> <a href="mailto:pulp-list@redhat.com" target="_blank">pulp-list@redhat.com</a><br>
<b>Subject:</b> Re: [Pulp-list] 2to3-migration takes a long time</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>Hey Adam,</div>
<div><br>
</div>
<div>It's not expected for subsequent migrations to take equally long, that is strange and concerning. This is with the very latest version of the migration plugin (0.5.0)? Is it actually equally long, or just not 'reasonably' faster? Is it only the "migrating
rpm content to Pulp 3 erratum" sub-item which reports doing a lot of work or are many of them doing so?<br>
</div>
<div><br>
</div>
<div>I do have some good news though, the 6-7 hour runtime will improve significantly once you can upgrade to 3.7.2 (which doesn't look like it has been pushed to the Foreman repositories yet). There was a significant performance regression introduced by 3.7.0
which has been fixed, and in my testing migrations only took a bit more than 1/3 as long as they had been taking previously.<br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, Oct 23, 2020 at 10:47 AM Winberg Adam <<a href="mailto:Adam.Winberg@smhi.se" target="_blank">Adam.Winberg@smhi.se</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div id="gmail-m_-1790087884150906635gmail-m_-5134876385173496077divtagdefaultwrapper" dir="ltr" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif">
<p>Depending on the size of your pulp2 installation I understand that the 2to3migration can take quite some time. In my environment it takes approx. 6-7hrs. However, I expect consecutive migrations to be faster, based on the documentation:</p>
<p>"<span>When you are ready to switch to Pulp 3: run migration, then stop Pulp 2 services (so no new data is coming in), run migration for the last time (it should not take long)."</span></p>
<p><span><br>
</span></p>
<p><span>But all my migration runs are equally long, the 'sub-task' "<span>Migrating rpm content to Pulp 3 erratum" processes about 90000 items every time which takes a long time. Is this expected behaviour?</span></span></p>
<p><span><span><br>
</span></span></p>
<p><span><span>Thanks, </span></span></p>
<p><span><span>Adam</span></span></p>
</div>
</div>
_______________________________________________<br>
Pulp-list mailing list<br>
<a href="mailto:Pulp-list@redhat.com" target="_blank">Pulp-list@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/pulp-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a></blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote></div>