[Pulp-list] epel syncs all advisories every time
Tanya Tereshchenko
ttereshc at redhat.com
Fri Feb 12 12:18:28 UTC 2021
Hi Adam,
Here is the issue to follow for the AdvisoryConflict
https://pulp.plan.io/issues/8249.
RPM folks discussed it in detail, and I believe this one should be fixed
soon.
There could be multiple reasons for advisory conflicts, but this specific
issue will likely cover your case.
I also filed a story https://pulp.plan.io/issues/8250 to cover other cases
for conflict resolution.
While your situation and a bug above ^^ is clearly a bug and there is a way
for Pulp to make a safe decision, with the remaining cases, only a user
could know for sure.
So we are not working on it and waiting if there is any demand for such a
feature. The solution won't be ideal, any automated conflict resolution for
the remaining cases would be under use-it-at-your-own-risk category.
Tanya
On Wed, Feb 10, 2021 at 3:34 PM Tanya Tereshchenko <ttereshc at redhat.com>
wrote:
> Great, it confirms that the advisory is recreated for updated ones only.
> Thanks for the feedback and for the filed issue.
>
> Tanya
>
> On Wed, Feb 10, 2021 at 3:18 PM Winberg Adam <Adam.Winberg at smhi.se> wrote:
>
>> > As for the recreating of advisories, if it happens to *all* of them
>> for every sync, it seems like a Pulp RPM bug to me.
>>
>> You are correct, it does not happen to all of them but the majority.
>>
>>
>> > If you try to sync two times in row, do you still see all advisories
>> being recreated?
>> No, two syncs in a row does not render any changes, it's only happening
>> on a new day (i.e. after epel repo metadata has been updated).
>>
>> I created an issue for this, https://pulp.plan.io/issues/8229, since I
>> don't think an advisory should be replaced in Pulp if the only changed
>> metadata is 'updated_date'. But I might not see the whole picture.
>>
>> //Adam
>>
>> ------------------------------
>> *From:* Tanya Tereshchenko <ttereshc at redhat.com>
>> *Sent:* 10 February 2021 14:45
>> *To:* Winberg Adam
>> *Cc:* pulp-list at redhat.com
>> *Subject:* Re: [Pulp-list] epel syncs all advisories every time
>>
>> Hi Adam,
>>
>> If you see this AdvisoryConflict for the migrated repos, please file an
>> issue https://pulp.plan.io/projects/migration/issues/new
>> Pulp2 managed advisories in a completely different way so there could be
>> pkglist discrepancies when trying to update them (because Fedora advisories
>> often refer to packages which are no longer in a repo)
>> We'll see if there is something we can suggest to fix those or if there
>> is a need to have a way in RPM plugin to overcome those in some user
>> friendly way.
>>
>> As for the recreating of advisories, if it happens to *all* of them for
>> every sync, it seems like a Pulp RPM bug to me.
>> If it happens to many of them and you sync once a day at most, it seems
>> to be related to what you pointed out - the updated date in EPEL repos
>> changes very often.
>> I checked EPEL7 and EPEL8 and indeed, not all but the majority of
>> advisories there have today as their updated date.
>> If you try to sync two times in row, do you still see all advisories
>> being recreated?
>>
>> Thanks,
>> Tanya
>>
>> On Mon, Feb 8, 2021 at 8:37 PM Winberg Adam <Adam.Winberg at smhi.se> wrote:
>>
>>> Looking a bit more on this the copy issue, with the error
>>>
>>>
>>> pulp_rpm.app.exceptions.AdvisoryConflict: Incoming and existing
>>> advisories have the same id and timestamp but different and intersecting
>>> package lists. At least one of them is wrong. Advisory id:
>>> FEDORA-EPEL-2019-927a9446df
>>>
>>> seems to be due to the advisory metadata in my frozen epel repos being
>>> broken, the package lists are incomplete. These repos are migrated from
>>> pulp2, but I don't know if the metadata was broken already in pulp2 or if
>>> it broke during the migration. My pulp2 environment is no longer with me so
>>> there's no way for me to find out.
>>>
>>> So I need to fix those repos. Once thats done the issue with EPEL syncs
>>> re-adding all advisories are probably not that big, but it does make
>>> syncing take quite some time and also makes it difficult for me to filter
>>> newly added advisories (since they are added at every sync). So if anyone
>>> has any idea how to solve this I would be grateful. Comparing the
>>> advisories that are added with the ones that are removed shows that they
>>> are identical except for a newer 'updated_date' on the added one (the
>>> updated_date is todays date).
>>>
>>> Maybe an advisory shouldnt be considered for addition if the only change
>>> is the updated_date and no other metadata is changed? And why is EPEL
>>> setting a new 'update_date' on their advisories at every rebuild/re-index
>>> of their repos?
>>>
>>> //Adam
>>>
>>>
>>> ------------------------------
>>> *From:* pulp-list-bounces at redhat.com <pulp-list-bounces at redhat.com> on
>>> behalf of Winberg Adam <Adam.Winberg at smhi.se>
>>> *Sent:* 08 February 2021 16:07
>>> *To:* pulp-list at redhat.com
>>> *Subject:* Re: [Pulp-list] epel syncs all advisories every time
>>>
>>>
>>> The patch was also for copying, not syncing, so I was totally confused.
>>> Sorry about that...
>>>
>>>
>>> ------------------------------
>>> *From:* pulp-list-bounces at redhat.com <pulp-list-bounces at redhat.com> on
>>> behalf of Winberg Adam <Adam.Winberg at smhi.se>
>>> *Sent:* 08 February 2021 15:58
>>> *To:* pulp-list at redhat.com
>>> *Subject:* Re: [Pulp-list] epel syncs all advisories every time
>>>
>>>
>>> Did a quick test and the patch i mentioned is not involved.
>>>
>>>
>>> Note that I only see this problem with EPEL, not with RHEL repos which
>>> also publishes erratas/advisories. So the problem is probably on the EPEL
>>> side, or..?
>>>
>>>
>>> //Adam
>>>
>>>
>>> ------------------------------
>>> *From:* pulp-list-bounces at redhat.com <pulp-list-bounces at redhat.com> on
>>> behalf of Winberg Adam <Adam.Winberg at smhi.se>
>>> *Sent:* 08 February 2021 10:55
>>> *To:* pulp-list at redhat.com
>>> *Subject:* Re: [Pulp-list] epel syncs all advisories every time
>>>
>>>
>>> I'm not sure, but I think this behaviour might have started after I
>>> applied the following patch:
>>>
>>>
>>> https://github.com/pulp/pulp_rpm/commit/1652026913308e8348543af6f62c3b5c5f89985b#diff-0b195d23762f04b205940bafb5889ddf96181afde122ead35f8c65fe03527647
>>>
>>>
>>> //Adam
>>>
>>>
>>> ------------------------------
>>> *From:* pulp-list-bounces at redhat.com <pulp-list-bounces at redhat.com> on
>>> behalf of Winberg Adam <Adam.Winberg at smhi.se>
>>> *Sent:* 08 February 2021 10:39
>>> *To:* pulp-list at redhat.com
>>> *Subject:* [Pulp-list] epel syncs all advisories every time
>>>
>>>
>>> I sync the rhel8 and rhel7 epel repo every day, and for some reason all
>>> advisories are removed and added each time:
>>>
>>>
>>> "content_summary": {
>>> "added": {
>>> "rpm.advisory": {
>>> "count": 2540,
>>> "href":
>>> "/pulp/api/v3/content/rpm/advisories/?repository_version_added=/pulp/api/v3/repositories/rpm/rpm/10e51ae6-65c7-42aa-8ab1-ffebdf752500/versions/19/"
>>> },
>>> "rpm.package": {
>>> "count": 6,
>>> "href":
>>> "/pulp/api/v3/content/rpm/packages/?repository_version_added=/pulp/api/v3/repositories/rpm/rpm/10e51ae6-65c7-42aa-8ab1-ffebdf752500/versions/19/"
>>> }
>>> },
>>> "removed": {
>>> "rpm.advisory": {
>>> "count": 2536,
>>> "href":
>>> "/pulp/api/v3/content/rpm/advisories/?repository_version_removed=/pulp/api/v3/repositories/rpm/rpm/10e51ae6-65c7-42aa-8ab1-ffebdf752500/versions/19/"
>>> }
>>> }
>>> },
>>>
>>>
>>> This also leads to problem when I want to copy new advisories to my
>>> frozen epel repo:
>>> pulp_rpm.app.exceptions.AdvisoryConflict: Incoming and existing
>>> advisories have the same id and timestamp but different and intersecting
>>> package lists. At least one of them is wrong. Advisory id:
>>> FEDORA-EPEL-2019-927a9446df
>>>
>>> So it seems that the advisory has actually changed, which seems weird.
>>> And changed every day? Is this some quirk in the EPEL sources or a bug
>>> in Pulp?
>>>
>>> //Adam
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Pulp-list mailing list
>>> Pulp-list at redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-list
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20210212/04303f5b/attachment.htm>
More information about the Pulp-list
mailing list