[Pulp-dev] versioned repositories

Fri May 26 19:48:04 UTC 2017

After some in-person convo, @mhrivnak and I agree the most productive next
step is for us to look at the API planning. We will continue with the
regularly scheduled use calls starting on Tuesday with the agenda 'status,
alternate content sources, and revisiting the JWT use cases based on
discussion from [0].

We also need to understand what the API looks like with versioned repos
fully implemented. We also need to understand what an API looks like that
allows us to launch the MVP without versioned repos and add them in later
without creating a backwards compatibility issue. This is also heavily
related to the nested views of DRF and the use of natural keys which are
topics others are more familiar with than I.

Here are some API options @mhrivnak and I talked about. Let .../foo/ be the
API path to the repository named foo. I'm assuming nested URLs.

# Sync
The sync trigger never deals with versioned repos because it creates
versions and doesn't refer to them.

.../foo/importers/<importer_natural_key>/sync

# Publish options

.../foo/publishers/<publisher_natural_key>/publish        <---- no version
specifies assumes latest. When versioned repos we can add a POST param
version=xxxx to adding this during 3.1+

.../foo/latest/publishers/<publisher_natural_key>/publish      <----
launching without versioned repos would cause 'latest' to be the only value
that wouldn't 404. When version repos are added, latest would be the
natural key of the repo version (probably it's number).

.../foo/versions/latest/publishers/<publisher_natural_key>/publish
<---- same as above only with the version repo being more of a first-class
resource.

# Listing Content options

.../foo/content/      <---- implicitly assumes the latest.
.../foo/versions/content/    <---- implicitly assumes the latest.
.../foo/versions/latest/content/   <---- explicitly latest. Launching
without versioned repos would cause 'latest' to be the only value that
wouldn't 404.

# Of all the ^, these make the most sense to me:

.../foo/importers/<importer_natural_key>/sync
.../foo/versions/<version_natural_key>/publishers/<publisher_natural_key>/publish
<---- hard code <version_natural_key> to be 'latest' for MVP and have no
internal implementation of versioned repos
.../foo/versions/<version_natural_key>/content/      <---- hard code
<version_natural_key> to be 'latest' for MVP and have no internal
implementation of versioned repos

Please send thoughts and idea on the API.

[0]: https://pulp.plan.io/issues/2359

-Brian

On Fri, May 26, 2017 at 11:58 AM, Michael Hrivnak <mhrivnak at redhat.com>
wrote:

>
> On Fri, May 26, 2017 at 10:31 AM, Brian Bouterse <bbouters at redhat.com>
> wrote:
>
>> +1 to determining the REST API because we have to get that right due to
>> semver.
>>
>> @ipanova we want the same things, but I'm not sure how to get there. You
>> are considering including all versioned repo functionality in the MVP. I
>> like that idea lot better than internally modelling it but not giving the
>> users the features. I really want versioned repos. In terms of how we get
>> there though, there are challenges in doing all of this for the MVP (see
>> below).
>>
>> @all, If we want to make decisions now on the data model, that is easier
>> said than done. If I thought we could easily agree on the data model, I
>> would think that path is more viable. There was a very long IRC
>> conversation a few days ago where an alternate data model was discussed and
>> concerns were raised with the prototype's data model in terms of
>> complexity. That led into a separate yet an equally long IRC convo about
>> the delete use cases of versioned repos. We can work through those
>> discussions, and in time we will, but it will take time. We also need to
>> involve the users, which will take time too.
>>
>
> Perhaps we should dig into the modeling options more. To me, it is clear
> that the simpler model, which has been considered and discussed in the
> past, is not viable because of the incredibly high number of relationships
> it would produce. The model I proposed is slightly more complex, but still
> easily understood and reasoned about. It's been vetted many times by the
> team both as a group and as individuals, so I don't think there's any
> particular risk with it. The PoC even includes correctness tests of all the
> behaviors I could think of.
>
> I appreciate all ideas and alternatives. We should compare and contrast
> all potential options. But I don't think this one suddenly needs to be a
> wrench in the gears.
>
> Does anyone have a preference on how to move this discussion forward? New
> email thread? Live meeting?
>
> I am admittedly coming at this with the bias of someone who has been
> thinking about and working on this problem for a long time. But that also
> gives me the confidence to know that we are very close to being able to
> move forward with implementation. I think we just need:
>
> - Final agreement on the model. I think we're not far from this.
> - Define basic use cases. Is there more to do here? I think this is well
> in-hand.
> - Make REST API decisions. This is more a matter of deciding on certain
> general approaches we will take with specific REST API challenges, such as
> how to make an endpoint that acts on multiple resources. There's not much
> specific to repo versions. It just happens to be the first such example
> we're discussing as a wide group.
>
>
>>
>> Pulp2 never had these features that Pulp3 can launch without them. I
>> really want versioned repos, but not at the expense of a longer Pulp3
>> release timeline. I can't stress enough how getting Pulp3 out needs to be
>> our focus and versioned repos (aside from the API decisions) are
>> unnecessarily extending the release timeline. I think we need to turn our
>> focus to the plugin conversion work.
>>
>
> The main reason I think this is important for 3.0 is that if it doesn't
> land there, it may not be viable until 4.0.
>
> And I really don't think this has to impact our timeline. It may actually
> be more work for 3.0 to have a REST API that's compatible with a data model
> we don't have in-place, using predictions of what it's likely to be, rather
> than just using the real models, serializers and views up-front. You all
> know how committed I am to deferring anything we can to 3.1+. But I see
> this one as very important to getting Pulp 3's foundation right.
>
> I do see this behavior as a huge advantage for Pulp and critical to the
> kinds of workflows we need to advocate and facilitate. If there's one
> critique we hear over and over, it's that promotions and small changes
> should be crazy fast. This is the path to making that a reality.
>
> --
>
> Michael Hrivnak
>
> Principal Software Engineer, RHCE
>
> Red Hat
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20170526/9bde3e65/attachment.htm>