[Pulp-dev] repository versions update

[Michael Hrivnak]

Following up on previous discussions, I did an analysis of how repository
versioning would impact Pulp 3's current REST API and plugin API. A lot has
changed since we last discussed the topic (in May 2017), such as how we
handle publications, and how the REST API is laid out. You can read the
analysis here:

https://pulp.plan.io/projects/pulp/wiki/Repository_Versions

We previously discussed and vetted the mechanics at great length. While
there was broad agreement on the value to Pulp 3, there was uncertainty
about the details of how it would impact REST clients and plugin writers,
and also uncertainty about how long it would take to fully implement.

In the course of my recent analysis, two things became clear. 1) both
current APIs are not compatible and would have to change. Details are on
the wiki page above. 2) the PoC from earlier this year indeed covers the
hard parts, leaving mostly DRF details to sort out.

I started rebasing the PoC onto current 3.0-dev, and within an hour I had
it working with the updated REST endpoints. With that having been so easy,
I threw caution to the wind, and within a few hours I had a fully
functional branch that covered all the key use cases.

- sync creates a new version
- versions and their content sets are visible through the REST API
- each version shows what content was added and removed
- versions can be deleted, which queues a task that squashes changes as
previously discussed
- the ChangeSet and pulp_file were updated to work with versions
- publish defaults to using the latest version

I also created a set of tests to help prove that it behaves correctly:

https://gist.github.com/mhrivnak/69af54063dff7465212914094dff34c2

I have just about 12 hours of recent work into it, and the code is
PR-ready. It's just missing doc updates and release notes. It's been
difficult to keep discussion moving toward a full plan due to the
uncertainties mentioned above, so hopefully this can alleviate those
concerns and give everyone something concrete to look at.

https://github.com/pulp/pulp/pull/3228
https://github.com/pulp/pulp_file/pull/20

Two notable items are missing. One is that there is no way to arbitrarily
add and remove content from a repo now, since this removes the
"repositorycontent" endpoint. But we need to solve that with a more formal
and bulk add/remove API anyway. I also found that the "repositorycontent"
endpoint was not using tasks, and thus there was no repo locking, so it
needed additional work anyway. Based on this overall effort, I think it
will be very easy to add if we just agree on what the endpoints should look
like.

The other is that publish does not in this PR accept a reference to a
version. It always uses the latest. That would also be a very easy
enhancement to make.

I am happy to support getting this merged as I transition to being a more
passive community member, assuming there are no objections. I am also of
course happy to help support this into the future, as I believe strongly in
its value and importance (see previous thread).

Please provide feedback and questions. If a live meeting this week would
help expedite evaluation of this effort, I'm happy to schedule that. And
assuming there are no hard objections, I'm happy to proceed with
documentation updates.

Thanks!

-- 

Michael Hrivnak

Principal Software Engineer, RHCE

Red Hat
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20171128/9eab713d/attachment.htm>