<div dir="ltr">Hi Adam,<div><br></div><div>Here is the issue to follow for the AdvisoryConflict <a href="https://pulp.plan.io/issues/8249">https://pulp.plan.io/issues/8249</a>.</div><div>RPM folks discussed it in detail, and I believe this one should be fixed soon. </div><div>There could be multiple reasons for advisory conflicts, but this specific issue will likely cover your case.</div><div><br></div><div>I also filed a story <a href="https://pulp.plan.io/issues/8250">https://pulp.plan.io/issues/8250</a> to cover other cases for conflict resolution. </div><div>While your situation and a bug above ^^ is clearly a bug and there is a way for Pulp to make a safe decision, with the remaining cases, only a user could know for sure.</div><div>So we are not working on it and waiting if there is any demand for such a feature. The solution won't be ideal, any automated conflict resolution for the remaining cases would be under use-it-at-your-own-risk category.</div><div><br></div><div>Tanya</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Feb 10, 2021 at 3:34 PM Tanya Tereshchenko <<a href="mailto:ttereshc@redhat.com">ttereshc@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Great, it confirms that the advisory is recreated for updated ones only.</div><div>Thanks for the feedback and for the filed issue.</div><div><br></div><div>Tanya</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Feb 10, 2021 at 3:18 PM Winberg Adam <<a href="mailto:Adam.Winberg@smhi.se" target="_blank">Adam.Winberg@smhi.se</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir="ltr"> <div id="gmail-m_-5350051633968909806gmail-m_4932135005717154240divtagdefaultwrapper" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif" dir="ltr"> <p>> <span>As for the recreating of advisories, if it happens to *all* of them for every sync, it seems like a Pulp RPM bug to me.</span></p> <p><span>You are correct, it does not happen to all of them but the majority. </span></p> <p><br> </p> <p>> <span>If you try to sync two times in row, do you still see all advisories being recreated?</span></p> No, two syncs in a row does not render any changes, it's only happening on a new day (i.e. after epel repo metadata has been updated). <div><br> </div> <div>I created an issue for this, <a href="https://pulp.plan.io/issues/8229" id="gmail-m_-5350051633968909806gmail-m_4932135005717154240LPlnk951111" target="_blank">https://pulp.plan.io/issues/8229</a>, since I don't think an advisory should be replaced in Pulp if the only changed metadata is 'updated_date'. But I might not see the whole picture. </div> <div><br> </div> <div>//Adam<br> <br> <div style="color:rgb(0,0,0)"> <hr style="display:inline-block;width:98%"> <div id="gmail-m_-5350051633968909806gmail-m_4932135005717154240divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Tanya Tereshchenko <<a href="mailto:ttereshc@redhat.com" target="_blank">ttereshc@redhat.com</a>><br> <b>Sent:</b> 10 February 2021 14:45<br> <b>To:</b> Winberg Adam<br> <b>Cc:</b> <a href="mailto:pulp-list@redhat.com" target="_blank">pulp-list@redhat.com</a><br> <b>Subject:</b> Re: [Pulp-list] epel syncs all advisories every time</font> <div> </div> </div> <div> <div dir="ltr">Hi Adam, <div><br> </div> <div>If you see this AdvisoryConflict for the migrated repos, please file an issue <a href="https://pulp.plan.io/projects/migration/issues/new" id="gmail-m_-5350051633968909806gmail-m_4932135005717154240LPlnk943034" target="_blank">https://pulp.plan.io/projects/migration/issues/new</a></div> <div>Pulp2 managed advisories in a completely different way so there could be pkglist discrepancies when trying to update them (because Fedora advisories often refer to packages which are no longer in a repo)</div> <div>We'll see if there is something we can suggest to fix those or if there is a need to have a way in RPM plugin to overcome those in some user friendly way.</div> <div><br> </div> <div>As for the recreating of advisories, if it happens to *all* of them for every sync, it seems like a Pulp RPM bug to me.</div> <div>If it happens to many of them and you sync once a day at most, it seems to be related to what you pointed out - the updated date in EPEL repos changes very often.</div> <div>I checked EPEL7 and EPEL8 and indeed, not all but the majority of advisories there have today as their updated date.</div> <div>If you try to sync two times in row, do you still see all advisories being recreated?</div> <div><br> </div> <div>Thanks,</div> <div>Tanya</div> </div> <br> <div class="gmail_quote"> <div dir="ltr" class="gmail_attr">On Mon, Feb 8, 2021 at 8:37 PM Winberg Adam <<a href="mailto:Adam.Winberg@smhi.se" target="_blank">Adam.Winberg@smhi.se</a>> wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir="ltr"> <div id="gmail-m_-5350051633968909806gmail-m_4932135005717154240gmail-m_-4545164307913833028divtagdefaultwrapper" dir="ltr" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif"> <p>Looking a bit more on this the copy issue, with the error</p> <p><br> </p> <p><span style="font-family:Consolas,Courier,monospace;font-size:8pt">pulp_rpm.app.exceptions.AdvisoryConflict: Incoming and existing advisories have the same id and timestamp but different and intersecting package lists. At least one of them is wrong. Advisory id: FEDORA-EPEL-2019-927a9446df</span><br> </p> <br> seems to be due to the advisory metadata in my frozen epel repos being broken, the package lists are incomplete. These repos are migrated from pulp2, but I don't know if the metadata was broken already in pulp2 or if it broke during the migration. My pulp2 environment is no longer with me so there's no way for me to find out. <div><br> </div> <div>So I need to fix those repos. Once thats done the issue with EPEL syncs re-adding all advisories are probably not that big, but it does make syncing take quite some time and also makes it difficult for me to filter newly added advisories (since they are added at every sync). So if anyone has any idea how to solve this I would be grateful. Comparing the advisories that are added with the ones that are removed shows that they are identical except for a newer 'updated_date' on the added one (the updated_date is todays date).</div> <div><br> </div> <div>Maybe an advisory shouldnt be considered for addition if the only change is the updated_date and no other metadata is changed? And why is EPEL setting a new 'update_date' on their advisories at every rebuild/re-index of their repos?</div> <div><br> </div> <div>//Adam</div> <div><br> <br> <div style="color:rgb(0,0,0)"> <hr style="display:inline-block;width:98%"> <div id="gmail-m_-5350051633968909806gmail-m_4932135005717154240gmail-m_-4545164307913833028divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> <a href="mailto:pulp-list-bounces@redhat.com" target="_blank">pulp-list-bounces@redhat.com</a> <<a href="mailto:pulp-list-bounces@redhat.com" target="_blank">pulp-list-bounces@redhat.com</a>> on behalf of Winberg Adam <<a href="mailto:Adam.Winberg@smhi.se" target="_blank">Adam.Winberg@smhi.se</a>><br> <b>Sent:</b> 08 February 2021 16:07<br> <b>To:</b> <a href="mailto:pulp-list@redhat.com" target="_blank">pulp-list@redhat.com</a><br> <b>Subject:</b> Re: [Pulp-list] epel syncs all advisories every time</font> <div> </div> </div> <div> <div id="gmail-m_-5350051633968909806gmail-m_4932135005717154240gmail-m_-4545164307913833028divtagdefaultwrapper" dir="ltr" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif"> <p>The patch was also for copying, not syncing, so I was totally confused. Sorry about that...</p> <br> <br> <div style="color:rgb(0,0,0)"> <hr style="display:inline-block;width:98%"> <div id="gmail-m_-5350051633968909806gmail-m_4932135005717154240gmail-m_-4545164307913833028divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> <a href="mailto:pulp-list-bounces@redhat.com" target="_blank">pulp-list-bounces@redhat.com</a> <<a href="mailto:pulp-list-bounces@redhat.com" target="_blank">pulp-list-bounces@redhat.com</a>> on behalf of Winberg Adam <<a href="mailto:Adam.Winberg@smhi.se" target="_blank">Adam.Winberg@smhi.se</a>><br> <b>Sent:</b> 08 February 2021 15:58<br> <b>To:</b> <a href="mailto:pulp-list@redhat.com" target="_blank">pulp-list@redhat.com</a><br> <b>Subject:</b> Re: [Pulp-list] epel syncs all advisories every time</font> <div> </div> </div> <div> <div id="gmail-m_-5350051633968909806gmail-m_4932135005717154240gmail-m_-4545164307913833028divtagdefaultwrapper" dir="ltr" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif"> <p>Did a quick test and the patch i mentioned is not involved. </p> <p><br> </p> <p>Note that I only see this problem with EPEL, not with RHEL repos which also publishes erratas/advisories. So the problem is probably on the EPEL side, or..?</p> <p><br> </p> <p>//Adam</p> <br> <br> <div style="color:rgb(0,0,0)"> <hr style="display:inline-block;width:98%"> <div id="gmail-m_-5350051633968909806gmail-m_4932135005717154240gmail-m_-4545164307913833028divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> <a href="mailto:pulp-list-bounces@redhat.com" target="_blank">pulp-list-bounces@redhat.com</a> <<a href="mailto:pulp-list-bounces@redhat.com" target="_blank">pulp-list-bounces@redhat.com</a>> on behalf of Winberg Adam <<a href="mailto:Adam.Winberg@smhi.se" target="_blank">Adam.Winberg@smhi.se</a>><br> <b>Sent:</b> 08 February 2021 10:55<br> <b>To:</b> <a href="mailto:pulp-list@redhat.com" target="_blank">pulp-list@redhat.com</a><br> <b>Subject:</b> Re: [Pulp-list] epel syncs all advisories every time</font> <div> </div> </div> <div> <div id="gmail-m_-5350051633968909806gmail-m_4932135005717154240gmail-m_-4545164307913833028divtagdefaultwrapper" dir="ltr" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif"> <p>I'm not sure, but I think this behaviour might have started after I applied the following patch:</p> <p><a href="https://github.com/pulp/pulp_rpm/commit/1652026913308e8348543af6f62c3b5c5f89985b#diff-0b195d23762f04b205940bafb5889ddf96181afde122ead35f8c65fe03527647" id="gmail-m_-5350051633968909806gmail-m_4932135005717154240gmail-m_-4545164307913833028LPlnk960876" target="_blank">https://github.com/pulp/pulp_rpm/commit/1652026913308e8348543af6f62c3b5c5f89985b#diff-0b195d23762f04b205940bafb5889ddf96181afde122ead35f8c65fe03527647</a><br> </p> <p><br> </p> <p>//Adam</p> <br> <br> <div style="color:rgb(0,0,0)"> <hr style="display:inline-block;width:98%"> <div id="gmail-m_-5350051633968909806gmail-m_4932135005717154240gmail-m_-4545164307913833028divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> <a href="mailto:pulp-list-bounces@redhat.com" target="_blank">pulp-list-bounces@redhat.com</a> <<a href="mailto:pulp-list-bounces@redhat.com" target="_blank">pulp-list-bounces@redhat.com</a>> on behalf of Winberg Adam <<a href="mailto:Adam.Winberg@smhi.se" target="_blank">Adam.Winberg@smhi.se</a>><br> <b>Sent:</b> 08 February 2021 10:39<br> <b>To:</b> <a href="mailto:pulp-list@redhat.com" target="_blank">pulp-list@redhat.com</a><br> <b>Subject:</b> [Pulp-list] epel syncs all advisories every time</font> <div> </div> </div> <div> <div id="gmail-m_-5350051633968909806gmail-m_4932135005717154240gmail-m_-4545164307913833028divtagdefaultwrapper" dir="ltr" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif"> <p>I sync the rhel8 and rhel7 epel repo every day, and for some reason all advisories are removed and added each time:</p> <p><br> </p> <p></p> <div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "content_summary": {</span></div> <div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "added": {</span></div> <div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "rpm.advisory": {</span></div> <div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "count": 2540,</span></div> <div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "href": "/pulp/api/v3/content/rpm/advisories/?repository_version_added=/pulp/api/v3/repositories/rpm/rpm/10e51ae6-65c7-42aa-8ab1-ffebdf752500/versions/19/"</span></div> <div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> },</span></div> <div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "rpm.package": {</span></div> <div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "count": 6,</span></div> <div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> "href": "/pulp/api/v3/content/rpm/packages/?repository_version_added=/pulp/api/v3/repositories/rpm/rpm/10e51ae6-65c7-42aa-8ab1-ffebdf752500/versions/19/"</span></div> <div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> }</span></div> <div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> },</span></div> <div><span style="font-family:Consolas,Courier,monospace;font-size:8pt"> <div> "removed": {</div> <div> "rpm.advisory": {</div> <div> "count": 2536,</div> <div> "href": "/pulp/api/v3/content/rpm/advisories/?repository_version_removed=/pulp/api/v3/repositories/rpm/rpm/10e51ae6-65c7-42aa-8ab1-ffebdf752500/versions/19/"</div> <div> }</div> <div> }</div> <div> },</div> <div><br> </div> </span></div> <div> <div><br> </div> <div><span style="font-size:12pt">This also leads to problem when I want to copy new advisories to my frozen epel repo:</span><br> </div> <div><span style="font-family:Consolas,Courier,monospace;font-size:8pt">pulp_rpm.app.exceptions.AdvisoryConflict: Incoming and existing advisories have the same id and timestamp but different and intersecting package lists. At least one of them is wrong. Advisory id: FEDORA-EPEL-2019-927a9446df</span><br> </div> <div><span><br> </span></div> <div><span>So it seems that the advisory has actually changed, which seems weird. And changed every day? </span>Is this some quirk in the EPEL sources or a bug in Pulp? </div> <div><br> </div> <div>//Adam</div> <div><span><br> </span></div> <div><br> </div> <br> </div> <br> <p></p> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> _______________________________________________<br> Pulp-list mailing list<br> <a href="mailto:Pulp-list@redhat.com" target="_blank">Pulp-list@redhat.com</a><br> <a href="https://www.redhat.com/mailman/listinfo/pulp-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a></blockquote> </div> </div> </div> </div> </div> </div> </blockquote></div></div> </blockquote></div>