[Pulp-dev] repository versions update
Jeff Ortel
jortel at redhat.com
Wed Nov 29 15:19:10 UTC 2017
On 11/29/2017 07:22 AM, David Davis wrote:
> I think we could design an API in 3.0 that would support versioned repos in 3.1+. However, our current API
> does not. For example, the /repositorycontents/ endpoint doesn't make sense with versioned repos as no one
> would want to add/remove content units one-by-one when doing so would generate a new repo version each time.
> Imagine that we end up with an endpoint in 3.0 that’s not compatible with versioned repos. What would we do? I
> think this is a strong argument for adding versioned repos now.
agreed.
>
> Of course the main drawback is that it might delay the beta. But I wonder by how much. It might be good to
> groom the versioned repo user stories so that (a) we can see how much value they provide to end users and (b)
> how closely they align with the work @mhrivnak has done.
agreed.
>
>
> David
>
> On Tue, Nov 28, 2017 at 4:00 PM, Brian Bouterse <bbouters at redhat.com <mailto:bbouters at redhat.com>> wrote:
>
> In reading back over the last email thread in May, it ended with us looking at URL options to ensure we
> could release 3.0 and add in repo versions in 3.1+. We definitely want repo versions in the 3.y line, so
> we wanted to make sure that was possible. If it wasn't, then we may have to add it into 3.0.
>
> That question is a lot easier now given how firm the API is. I think we can add in versioned repos in
> 3.1+, in a natural way. Just like a user creates a Publication which triggers a publish, a user would
> create a RepoVersion which would trigger a sync to produce that new RepoVersion. The repo versions work
> needs to continue, but first I hope we prioritize getting to Beta 1 for core. There are a lot of use cases
> in black on the MVP which are not implemented or written in Redmine. I believe closing that gap would be a
> better use of time given that we can add this later.
>
> What do others think?
>
>
> On Tue, Nov 28, 2017 at 2:24 PM, Dennis Kliban <dkliban at redhat.com <mailto:dkliban at redhat.com>> wrote:
>
> I have a hard objection to including versioned repositories in 3.0. We agreed to make sure that our
> current design would not prevent us from adding versioned repositories in the future. We did NOT agree
> to including versioned repositories in 3.0 release. This is a big code change that did not go through
> our regular planning process. I greatly appreciate your effort in driving this feature forward, but we
> should take a step back and go through our regular process. I am also concerned that adding such a big
> change at this time will delay the beta.
>
> -Dennis
>
>
> On Tue, Nov 28, 2017 at 10:10 AM, Michael Hrivnak <mhrivnak at redhat.com <mailto:mhrivnak at redhat.com>>
> wrote:
>
> Following up on previous discussions, I did an analysis of how repository versioning would impact
> Pulp 3's current REST API and plugin API. A lot has changed since we last discussed the topic (in
> May 2017), such as how we handle publications, and how the REST API is laid out. You can read the
> analysis here:
>
> https://pulp.plan.io/projects/pulp/wiki/Repository_Versions
> <https://pulp.plan.io/projects/pulp/wiki/Repository_Versions>
>
> We previously discussed and vetted the mechanics at great length. While there was broad agreement
> on the value to Pulp 3, there was uncertainty about the details of how it would impact REST
> clients and plugin writers, and also uncertainty about how long it would take to fully implement.
>
> In the course of my recent analysis, two things became clear. 1) both current APIs are not
> compatible and would have to change. Details are on the wiki page above. 2) the PoC from earlier
> this year indeed covers the hard parts, leaving mostly DRF details to sort out.
>
>
> I don't agree with your assessment that the current REST API is not compatible with adding repository
> versions. A repository version is it's own resource that can be added
>
>
>
> I started rebasing the PoC onto current 3.0-dev, and within an hour I had it working with the
> updated REST endpoints. With that having been so easy, I threw caution to the wind, and within a
> few hours I had a fully functional branch that covered all the key use cases.
>
> - sync creates a new version
> - versions and their content sets are visible through the REST API
> - each version shows what content was added and removed
> - versions can be deleted, which queues a task that squashes changes as previously discussed
> - the ChangeSet and pulp_file were updated to work with versions
> - publish defaults to using the latest version
>
> I also created a set of tests to help prove that it behaves correctly:
>
> https://gist.github.com/mhrivnak/69af54063dff7465212914094dff34c2
> <https://gist.github.com/mhrivnak/69af54063dff7465212914094dff34c2>
>
> I have just about 12 hours of recent work into it, and the code is PR-ready. It's just missing doc
> updates and release notes. It's been difficult to keep discussion moving toward a full plan due to
> the uncertainties mentioned above, so hopefully this can alleviate those concerns and give
> everyone something concrete to look at.
>
> https://github.com/pulp/pulp/pull/3228 <https://github.com/pulp/pulp/pull/3228>
> https://github.com/pulp/pulp_file/pull/20 <https://github.com/pulp/pulp_file/pull/20>
>
> Two notable items are missing. One is that there is no way to arbitrarily add and remove content
> from a repo now, since this removes the "repositorycontent" endpoint. But we need to solve that
> with a more formal and bulk add/remove API anyway. I also found that the "repositorycontent"
> endpoint was not using tasks, and thus there was no repo locking, so it needed additional work
> anyway. Based on this overall effort, I think it will be very easy to add if we just agree on what
> the endpoints should look like.
>
> The other is that publish does not in this PR accept a reference to a version. It always uses the
> latest. That would also be a very easy enhancement to make.
>
> I am happy to support getting this merged as I transition to being a more passive community
> member, assuming there are no objections. I am also of course happy to help support this into the
> future, as I believe strongly in its value and importance (see previous thread).
>
> Please provide feedback and questions. If a live meeting this week would help expedite evaluation
> of this effort, I'm happy to schedule that. And assuming there are no hard objections, I'm happy
> to proceed with documentation updates.
>
> Thanks!
>
> --
>
> Michael Hrivnak
>
> Principal Software Engineer, RHCE
>
> Red Hat
>
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com <mailto:Pulp-dev at redhat.com>
> https://www.redhat.com/mailman/listinfo/pulp-dev <https://www.redhat.com/mailman/listinfo/pulp-dev>
>
>
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com <mailto:Pulp-dev at redhat.com>
> https://www.redhat.com/mailman/listinfo/pulp-dev <https://www.redhat.com/mailman/listinfo/pulp-dev>
>
>
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com <mailto:Pulp-dev at redhat.com>
> https://www.redhat.com/mailman/listinfo/pulp-dev <https://www.redhat.com/mailman/listinfo/pulp-dev>
>
>
>
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 847 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20171129/4ce91379/attachment.sig>
More information about the Pulp-dev
mailing list