[Pulp-dev] repository versions update

David Davis daviddavis at redhat.com
Wed Nov 29 13:22:08 UTC 2017

I think we could design an API in 3.0 that would support versioned repos in
3.1+. However, our current API does not. For example, the
/repositorycontents/ endpoint doesn't make sense with versioned repos as no
one would want to add/remove content units one-by-one when doing so would
generate a new repo version each time. Imagine that we end up with an
endpoint in 3.0 that’s not compatible with versioned repos. What would we
do? I think this is a strong argument for adding versioned repos now.

Of course the main drawback is that it might delay the beta. But I wonder
by how much. It might be good to groom the versioned repo user stories so
that (a) we can see how much value they provide to end users and (b) how
closely they align with the work @mhrivnak has done.


On Tue, Nov 28, 2017 at 4:00 PM, Brian Bouterse <bbouters at redhat.com> wrote:

> In reading back over the last email thread in May, it ended with us
> looking at URL options to ensure we could release 3.0 and add in repo
> versions in 3.1+. We definitely want repo versions in the 3.y line, so we
> wanted to make sure that was possible. If it wasn't, then we may have to
> add it into 3.0.
> That question is a lot easier now given how firm the API is. I think we
> can add in versioned repos in 3.1+, in a natural way. Just like a user
> creates a Publication which triggers a publish, a user would create a
> RepoVersion which would trigger a sync to produce that new RepoVersion. The
> repo versions work needs to continue, but first I hope we prioritize
> getting to Beta 1 for core. There are a lot of use cases in black on the
> MVP which are not implemented or written in Redmine. I believe closing that
> gap would be a better use of time given that we can add this later.
> What do others think?
> On Tue, Nov 28, 2017 at 2:24 PM, Dennis Kliban <dkliban at redhat.com> wrote:
>> I have a hard objection to including versioned repositories in 3.0. We
>> agreed to make sure that our current design would not prevent us from
>> adding versioned repositories in the future. We did NOT agree to including
>> versioned repositories in 3.0 release. This is a big code change that did
>> not go through our regular planning process. I greatly appreciate your
>> effort in driving this feature forward, but we should take a step back and
>> go through our regular process. I am also concerned that adding such a big
>> change at this time will delay the beta.
>> -Dennis
>> On Tue, Nov 28, 2017 at 10:10 AM, Michael Hrivnak <mhrivnak at redhat.com>
>> wrote:
>>> Following up on previous discussions, I did an analysis of how
>>> repository versioning would impact Pulp 3's current REST API and plugin
>>> API. A lot has changed since we last discussed the topic (in May 2017),
>>> such as how we handle publications, and how the REST API is laid out. You
>>> can read the analysis here:
>>> https://pulp.plan.io/projects/pulp/wiki/Repository_Versions
>>> We previously discussed and vetted the mechanics at great length. While
>>> there was broad agreement on the value to Pulp 3, there was uncertainty
>>> about the details of how it would impact REST clients and plugin writers,
>>> and also uncertainty about how long it would take to fully implement.
>>> In the course of my recent analysis, two things became clear. 1) both
>>> current APIs are not compatible and would have to change. Details are on
>>> the wiki page above. 2) the PoC from earlier this year indeed covers the
>>> hard parts, leaving mostly DRF details to sort out.
>> I don't agree with your assessment that the current REST API is not
>> compatible with adding repository versions. A repository version is it's
>> own resource that can be added
>>> I started rebasing the PoC onto current 3.0-dev, and within an hour I
>>> had it working with the updated REST endpoints. With that having been so
>>> easy, I threw caution to the wind, and within a few hours I had a fully
>>> functional branch that covered all the key use cases.
>>> - sync creates a new version
>>> - versions and their content sets are visible through the REST API
>>> - each version shows what content was added and removed
>>> - versions can be deleted, which queues a task that squashes changes as
>>> previously discussed
>>> - the ChangeSet and pulp_file were updated to work with versions
>>> - publish defaults to using the latest version
>>> I also created a set of tests to help prove that it behaves correctly:
>>> https://gist.github.com/mhrivnak/69af54063dff7465212914094dff34c2
>>> I have just about 12 hours of recent work into it, and the code is
>>> PR-ready. It's just missing doc updates and release notes. It's been
>>> difficult to keep discussion moving toward a full plan due to the
>>> uncertainties mentioned above, so hopefully this can alleviate those
>>> concerns and give everyone something concrete to look at.
>>> https://github.com/pulp/pulp/pull/3228
>>> https://github.com/pulp/pulp_file/pull/20
>>> Two notable items are missing. One is that there is no way to
>>> arbitrarily add and remove content from a repo now, since this removes the
>>> "repositorycontent" endpoint. But we need to solve that with a more formal
>>> and bulk add/remove API anyway. I also found that the "repositorycontent"
>>> endpoint was not using tasks, and thus there was no repo locking, so it
>>> needed additional work anyway. Based on this overall effort, I think it
>>> will be very easy to add if we just agree on what the endpoints should look
>>> like.
>>> The other is that publish does not in this PR accept a reference to a
>>> version. It always uses the latest. That would also be a very easy
>>> enhancement to make.
>>> I am happy to support getting this merged as I transition to being a
>>> more passive community member, assuming there are no objections. I am also
>>> of course happy to help support this into the future, as I believe strongly
>>> in its value and importance (see previous thread).
>>> Please provide feedback and questions. If a live meeting this week would
>>> help expedite evaluation of this effort, I'm happy to schedule that. And
>>> assuming there are no hard objections, I'm happy to proceed with
>>> documentation updates.
>>> Thanks!
>>> --
>>> Michael Hrivnak
>>> Principal Software Engineer, RHCE
>>> Red Hat
>>> _______________________________________________
>>> Pulp-dev mailing list
>>> Pulp-dev at redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>> _______________________________________________
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20171129/db2c1eef/attachment.htm>

More information about the Pulp-dev mailing list