[Pulp-list] rpm plugin: multirepo copy of errata not working

Wed Sep 30 07:11:13 UTC 2020

>Ok, tried that now, synced a total of 72 advisories in 3-4 copy operations.

This was a bit unclear I realize, but I created new pulp3 repos for appstream and baseos and synced them with our redhat cdn remote as source. Then I _copied_ a total of 72 advisories from this new appstream repo to a new empty repo, while using the new baseos repo as a 'dependency source' paired with another new empty repo.

//Adam

________________________________
From: pulp-list-bounces at redhat.com <pulp-list-bounces at redhat.com> on behalf of Winberg Adam <Adam.Winberg at smhi.se>
Sent: 30 September 2020 09:06
To: Daniel Alley
Cc: pulp-list at redhat.com
Subject: Re: [Pulp-list] rpm plugin: multirepo copy of errata not working

We're glad to help, Pulp is extremely useful for us so if we can contribute we are happy to do so. I created an issue for this:

https://pulp.plan.io/issues/7625

>  If you sync the Pulp 3 repository using the migrated remote, what happens?

Do you mean syncing the 'my-new-repo1' repo with the migrated appstream remote?

> Are the Pulp 2 repositories you're migrating modified significantly from when they were originally synced?
Well, in the case of the appstream repo it has been synced against the Red Hat source every night, so the content has certainly been modified. Otherwise than that, no.

> If you make brand new Pulp 3 repositories and try the same copy operation, does the same weird copy behavior occur?
Do you mean new pulp3 repos synced using the appstream and baseos remotes (w. content from redhat cdn) and then using that as the source of the copy? Ok, tried that now, synced a total of 72 advisories in 3-4 copy operations. The copy itself between my new pulp3 appstream repo and the new empty repo works well and 610 rpm packages has been copied. But there are no dependencies copied from the new baseos repo, no content at all.

//Adam

________________________________
From: Daniel Alley <dalley at redhat.com>
Sent: 29 September 2020 20:28
To: Winberg Adam
Cc: pulp-list at redhat.com
Subject: Re: [Pulp-list] rpm plugin: multirepo copy of errata not working

Hi Adam,

Thank you for providing feedback on some of these rough edges, it is extremely helpful.

There's a couple of things you can try that would give us some useful information.  If you sync the Pulp 3 repository using the migrated remote, what happens?  Does the repository change or does it stay the same?

Are the Pulp 2 repositories you're migrating modified significantly from when they were originally synced? If you make brand new Pulp 3 repositories and try the same copy operation, does the same weird copy behavior occur?

Also, if it wouldn't be too much trouble, could you file this as an issue on our bug tracker (https://pulp.plan.io/) with the details?  We can continue discussion here but if there is a lot of information it's better to keep it in one place on the issue.

Thanks again,
Daniel

On Tue, Sep 29, 2020 at 8:21 AM Winberg Adam <Adam.Winberg at smhi.se<mailto:Adam.Winberg at smhi.se>> wrote:

I applied the patch from

https://github.com/pulp/pulp_rpm/commit/712abdf1abb95c969b54fd2968a573189b77bcba

and the the copy then went through without errors.

I'm a bit confused by the result however. I copied 16 advisories from the appstreams repo to my new empty repo and the copy ended up with copying all modulemds and almost all packages (11466 of 12053). That doesnt seem right to me. And my other new repo ('my-new-repo2') is still empty, meaning that of those 11000 packages there were none that had any dependencies from the baseos repo which also strikes me as odd. If i set 'dependency_solving=True' (contrary to the documentation) I end up with the same amount of packages in my 'my-new-repo1' and 2 packages in 'my-new-repo2'.

Am I misunderstanding something about this functionality?

//Adam

________________________________
From: Winberg Adam
Sent: 28 September 2020 16:44
To: pulp-list at redhat.com<mailto:pulp-list at redhat.com>
Subject: rpm plugin: multirepo copy of errata not working

I have succeeded in migrating my pulp2 content to pulp3 and all repos look complete (i've reran the migration and no new repoversions are generated).

In an attempt to test the multirepo copy functionality described at

https://pulp-rpm.readthedocs.io/en/latest/workflows/copy.html#recipes

, I created a couple of new, empty repos and tried to copy errata from my migrated RHEL8-appstream repo to them:

POST /pulp/api/v3/rpm/copy/
config:=[
    {"source_repo_version": "<rhel8-appstream-version-href>", "dest_repo": "my-new-repo1", "content": [$ADVISORY_HREF1]},
    {"source_repo_version": "<rhel8-baseos-version-href>", "dest_repo": "my-new-repo2", "content": []},
]
dependency_solving=False

All looks correct, but the operation ultimately fails with the following error:

        "description": "Modulemd matching query does not exist.",
        "traceback": "  File \"/usr/lib/python3.6/site-packages/rq/worker.py\", line 883, in perform_job\n    rv = job.perform()\n  File \"/usr/lib/python3.6/site-packages/rq/job.py\", line 657, in perform\n    self._result = self._execute()\n  File \"/usr/lib/python3.6/site-packages/rq/job.py\", line 663, in _execute\n    return self.func(*self.args, **self.kwargs)\n  File \"/usr/lib64/python3.6/contextlib.py\", line 52, in inner\n    return func(*args, **kwds)\n  File \"/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/copy.py\", line 167, in copy_content\n    content_to_copy |= find_children_of_content(content_to_copy, source_repo_version)\n  File \"/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/copy.py\", line 74, in find_children_of_content\n    name=name, stream=stream, version=version, context=context, arch=arch)\n  File \"/usr/lib/python3.6/site-packages/django/db/models/query.py\", line 408, in get\n    self.model._meta.object_name\n"
    },

Any ideas why this happens?

Regards,

Adam

_______________________________________________
Pulp-list mailing list
Pulp-list at redhat.com<mailto:Pulp-list at redhat.com>
https://www.redhat.com/mailman/listinfo/pulp-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20200930/e92b7f51/attachment.htm>