[Pulp-list] 2to3-migration takes a long time

Daniel Alley dalley at redhat.com
Fri Oct 23 17:02:46 UTC 2020


>
> No, this is with 0.4.0, since 0.5.0 is quite new I haven't updated yet.
> Will do so and try again.
>

If it's 0.4.0, then this is actually not surprising after all.  This was a
bug that was fixed in 0.5.0

https://pulp.plan.io/issues/7280

On Fri, Oct 23, 2020 at 12:45 PM Winberg Adam <Adam.Winberg at smhi.se> wrote:

> > This is with the very latest version of the migration plugin (0.5.0)?
> No, this is with 0.4.0, since 0.5.0 is quite new I haven't updated yet.
> Will do so and try again.
>
> > Is it actually equally long, or just not 'reasonably' faster? Is it
> only the "migrating rpm content to Pulp 3 erratum" sub-item which reports
> doing a lot of work or are many of them doing so?
> I would say that the runtime is very similar. And it looks like many if
> not all sub-items are repeating their work but since the 'pulp3 erratum' is
> the longest running that's the one that stands out. Here are some progress
> reports on a subsequent run which should only resulted in a handful of new
> rpm's:
>     {
>       "code": "migrating.rpm.content",
>       "done": 32348,
>       "message": "Migrating rpm content to Pulp 3 rpm",
>       "state": "completed",
>       "suffix": null,
>       "total": 32348
>     },
>     {
>       "code": "premigrating.content.general",
>       "done": 89690,
>       "message": "Pre-migrating Pulp 2 ERRATUM content (general info)",
>       "state": "completed",
>       "suffix": null,
>       "total": 89690
>     },
>     {
>       "code": "premigrating.content.general",
>       "done": 39,
>       "message": "Pre-migrating Pulp 2 RPM content (general info)",
>       "state": "completed",
>       "suffix": null,
>       "total": 43
>     },
>     {
>       "code": "migrating.rpm.content",
>       "done": 89690,
>       "message": "Migrating rpm content to Pulp 3 erratum",
>       "state": "completed",
>       "suffix": null,
>       "total": 89690
>     },
>
> Should these numbers be 0 on a subsequent run if nothing has changed?
>
> //Adam
>
>
> ------------------------------
> *From:* Daniel Alley <dalley at redhat.com>
> *Sent:* 23 October 2020 17:33
> *To:* Winberg Adam
> *Cc:* pulp-list at redhat.com
> *Subject:* Re: [Pulp-list] 2to3-migration takes a long time
>
> Hey Adam,
>
> It's not expected for subsequent migrations to take equally long, that is
> strange and concerning.  This is with the very latest version of the
> migration plugin (0.5.0)?  Is it actually equally long, or just not
> 'reasonably' faster? Is it only the "migrating rpm content to Pulp 3
> erratum" sub-item which reports doing a lot of work or are many of them
> doing so?
>
> I do have some good news though, the 6-7 hour runtime will improve
> significantly once you can upgrade to 3.7.2 (which doesn't look like it has
> been pushed to the Foreman repositories yet).  There was a significant
> performance regression introduced by 3.7.0 which has been fixed, and in my
> testing migrations only took a bit more than 1/3 as long as they had been
> taking previously.
>
> On Fri, Oct 23, 2020 at 10:47 AM Winberg Adam <Adam.Winberg at smhi.se>
> wrote:
>
>> Depending on the size of your pulp2 installation I understand that the
>> 2to3migration can take quite some time. In my environment it takes approx.
>> 6-7hrs. However, I expect consecutive migrations to be faster, based on the
>> documentation:
>>
>> "When you are ready to switch to Pulp 3: run migration, then stop Pulp 2
>> services (so no new data is coming in), run migration for the last time (it
>> should not take long)."
>>
>>
>> But all my migration runs are equally long, the 'sub-task' "Migrating
>> rpm content to Pulp 3 erratum" processes about 90000 items every time which
>> takes a long time. Is this expected behaviour?
>>
>>
>> Thanks,
>>
>> Adam
>> _______________________________________________
>> Pulp-list mailing list
>> Pulp-list at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-list
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20201023/b083f420/attachment.htm>


More information about the Pulp-list mailing list