[Pulp-list] 2to3-migration takes a long time
Daniel Alley
dalley at redhat.com
Fri Oct 23 17:02:46 UTC 2020
>
> No, this is with 0.4.0, since 0.5.0 is quite new I haven't updated yet.
> Will do so and try again.
>
If it's 0.4.0, then this is actually not surprising after all. This was a
bug that was fixed in 0.5.0
https://pulp.plan.io/issues/7280
On Fri, Oct 23, 2020 at 12:45 PM Winberg Adam <Adam.Winberg at smhi.se> wrote:
> > This is with the very latest version of the migration plugin (0.5.0)?
> No, this is with 0.4.0, since 0.5.0 is quite new I haven't updated yet.
> Will do so and try again.
>
> > Is it actually equally long, or just not 'reasonably' faster? Is it
> only the "migrating rpm content to Pulp 3 erratum" sub-item which reports
> doing a lot of work or are many of them doing so?
> I would say that the runtime is very similar. And it looks like many if
> not all sub-items are repeating their work but since the 'pulp3 erratum' is
> the longest running that's the one that stands out. Here are some progress
> reports on a subsequent run which should only resulted in a handful of new
> rpm's:
> {
> "code": "migrating.rpm.content",
> "done": 32348,
> "message": "Migrating rpm content to Pulp 3 rpm",
> "state": "completed",
> "suffix": null,
> "total": 32348
> },
> {
> "code": "premigrating.content.general",
> "done": 89690,
> "message": "Pre-migrating Pulp 2 ERRATUM content (general info)",
> "state": "completed",
> "suffix": null,
> "total": 89690
> },
> {
> "code": "premigrating.content.general",
> "done": 39,
> "message": "Pre-migrating Pulp 2 RPM content (general info)",
> "state": "completed",
> "suffix": null,
> "total": 43
> },
> {
> "code": "migrating.rpm.content",
> "done": 89690,
> "message": "Migrating rpm content to Pulp 3 erratum",
> "state": "completed",
> "suffix": null,
> "total": 89690
> },
>
> Should these numbers be 0 on a subsequent run if nothing has changed?
>
> //Adam
>
>
> ------------------------------
> *From:* Daniel Alley <dalley at redhat.com>
> *Sent:* 23 October 2020 17:33
> *To:* Winberg Adam
> *Cc:* pulp-list at redhat.com
> *Subject:* Re: [Pulp-list] 2to3-migration takes a long time
>
> Hey Adam,
>
> It's not expected for subsequent migrations to take equally long, that is
> strange and concerning. This is with the very latest version of the
> migration plugin (0.5.0)? Is it actually equally long, or just not
> 'reasonably' faster? Is it only the "migrating rpm content to Pulp 3
> erratum" sub-item which reports doing a lot of work or are many of them
> doing so?
>
> I do have some good news though, the 6-7 hour runtime will improve
> significantly once you can upgrade to 3.7.2 (which doesn't look like it has
> been pushed to the Foreman repositories yet). There was a significant
> performance regression introduced by 3.7.0 which has been fixed, and in my
> testing migrations only took a bit more than 1/3 as long as they had been
> taking previously.
>
> On Fri, Oct 23, 2020 at 10:47 AM Winberg Adam <Adam.Winberg at smhi.se>
> wrote:
>
>> Depending on the size of your pulp2 installation I understand that the
>> 2to3migration can take quite some time. In my environment it takes approx.
>> 6-7hrs. However, I expect consecutive migrations to be faster, based on the
>> documentation:
>>
>> "When you are ready to switch to Pulp 3: run migration, then stop Pulp 2
>> services (so no new data is coming in), run migration for the last time (it
>> should not take long)."
>>
>>
>> But all my migration runs are equally long, the 'sub-task' "Migrating
>> rpm content to Pulp 3 erratum" processes about 90000 items every time which
>> takes a long time. Is this expected behaviour?
>>
>>
>> Thanks,
>>
>> Adam
>> _______________________________________________
>> Pulp-list mailing list
>> Pulp-list at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-list
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20201023/b083f420/attachment.htm>
More information about the Pulp-list
mailing list