[Pulp-dev] Pulp 3 REST API Challenges
Jeff Ortel
jortel at redhat.com
Fri Apr 13 19:55:59 UTC 2018
On 04/12/2018 04:49 PM, Dennis Kliban wrote:
> On Thu, Apr 12, 2018 at 2:49 PM, Jeff Ortel <jortel at redhat.com
> <mailto:jortel at redhat.com>> wrote:
>
>
>
> On 04/11/2018 01:13 PM, Dennis Kliban wrote:
>> On Tue, Apr 10, 2018 at 6:44 PM, Jeff Ortel <jortel at redhat.com
>> <mailto:jortel at redhat.com>> wrote:
>>
>>
>>
>> On 04/10/2018 04:15 PM, Dennis Kliban wrote:
>>> On Tue, Apr 10, 2018 at 2:04 PM, Brian Bouterse
>>> <bbouters at redhat.com <mailto:bbouters at redhat.com>> wrote:
>>>
>>> These are good problem statements. I didn't understand
>>> all of the aspects of it, so I put some inline questions.
>>>
>>> My overall question is: are these related problems? To
>>> share my answer to this, I believe the first two
>>> problems are related and the third is separate. The
>>> classic divide and conquor approach we could use here is
>>> to confirm that the problems are unrelated and focus on
>>> resolving one of them first.
>>>
>>>
>>> I don't think all 3 are related problems. The motivation for
>>> grouping all together is that a subset of the action
>>> endpoints from problem 1 are used to create repository
>>> versions and Problem 3 is a problem with the repository
>>> version creation API.
>>>
>>>
>>> On Mon, Apr 9, 2018 at 3:18 PM, Austin Macdonald
>>> <austin at redhat.com <mailto:austin at redhat.com>> wrote:
>>>
>>> Folks,
>>>
>>> Austin, Dennis, and Milan have identified the
>>> following issues with current Pulp3 REST API design:
>>>
>>> * Action endpoints are problematic.
>>> o Example POST@/importers/<plugin>/sync/
>>> o They are non-RESTful and would make client
>>> code tightly coupled with the server code.
>>> o These endpoints are inconsistent with the
>>> other parts of the REST API.
>>>
>>> Is self-consistency really a goal? I think it's a
>>> placeholder for consistency for REST since the "rest" of
>>> the API is RESTful. After reading parts of Roy
>>> Fielding's writeup of the definition of REST I believe
>>> "action endpoints are not RESTful" to be a true
>>> statement. Maybe "Action endpoints are problematic"
>>> should be replaced with "Action endpoints are not
>>> RESTful" perhaps and have the self-consistency bullet
>>> removed?
>>>
>>>
>>> +1 to "Action endpoints are not RESTful"
>>> +1 to removing the self-consistency language
>>>
>>> o DRF is not being used as intended for action
>>> endpoints so we have to implement extra
>>> code. (against the grain)
>>>
>>> I don't know much about this. Where is the extra code?
>>>
>>> * We don't have a convention for where
>>> plug-in-specific, custom repository version
>>> creation endpoints.
>>> o example POST@/api/v3/<where?>/docker/add/
>>> o needs to be discoverable through the schema
>>>
>>> What does discoverable via the schema ^ mean? Aren't all
>>> urls listed in the schema?
>>>
>>> I think of ^ problem somewhat differently. Yes all urls
>>> need to be discoverable (a REST property), but isn't it
>>> more of an issue that the urls which produce repo
>>> versions can't be identified distinctly from any other
>>> plugin-contributed url? To paraphrase this perspective:
>>> making a repo version is strewn about throughout the API
>>> in random places which is a bad user experience. Is that
>>> what is motivation url discovery?
>>>
>>>
>>> Yes. I envision a CLI that can discover new plugin
>>> repository-version-creating functionality without having to
>>> install new client packages. Allowing plugin writers to add
>>> endpoints in arbitrary places for creating repository
>>> versions will make it impossible for the client to know what
>>> all the possible ways of creating a repository version are.
>>>
>>> * For direct repository version creation, plugins
>>> are not involved.
>>> o validation correctness problem:
>>> https://pulp.plan.io/issues/3541
>>> <https://pulp.plan.io/issues/3541>
>>> o example:
>>> POST@/api/v3/repositories/<repository_id>/versions/
>>>
>>> I agree with this problem statement. In terms of scope
>>> it affects some plugin writers but not all.
>>>
>>>
>>> I think it affects all plugin writers. Even the File plugin
>>> needs to provide some validation when creating a repository
>>> version. Right now you can add a FileContent with the same
>>> relative path as another FileContent in the repository
>>> version. This should not be possible because it's not a
>>> valid combination of FileContent units in the same
>>> repository version.
>>
>> Not necessarily. Two files with the same relative path will
>> have different digest (different content). The assumption
>> that they both cannot be in the same repository makes
>> assumptions about how the repository is used which I don't
>> think is a good idea. Image two different versions of abc.iso.
>>
>>
>> Why is it bad to assume that a repository version is going to be
>> published? What are the other ways to use a repository version?
>
> The repository may not be publish and/or may not be published by
> the FilePublisher in the file plugin project. A user may want to
> sync and store many version of an iso in the repository and then
> selectively /add/ a specific version to another repository for
> promotion work flows. Also, the user could use another (custom)
> publisher that deals differently with multiple files with the same
> path in the repository. The FilePublisher currently publishes the
> newest. My point being, we, really cannot assume how the
> repository will be used or which publisher /may/ publish it.
>
>
> The problem was initially stated as "For direct repository version
> creation, plugins are not involved". It sounds like you disagree that
> this is a problem.
Yes. Definitely, agreed.
> Can you confirm this by telling us if plugins should be able to
> provide validation for this API provided by core?
Plugins participating in core endpoints is different, broader discussion.
The following is not aimed at you dkliban :)
We need to decide if we want to return to the pulp2 pattern whereby the
core delegates behavior to plugins via the plugin API. Or, continue
down the pulp3 path whereby operations involving plugins are contributed
to the API by each plugin (not making a value judgment). Also, I value
consistency in APIs and don't think these approaches should be mixed
(with the exception of content related live-API). Consistency in APIs
reflect both a thoughtful, mature design and provides a better user
experience. I'm sure everyone has cursed APIs that did things
every-which-way. I don't think there is any difference between creating
a repository version via sync or creating a version with a list of
content to add/remove. And to a lesser degree publishing. We should
either POST to the /publications/ endpoint for creating a publication
(core API), _or_ users should POST to the plugin contributed endpoint
(as currently) for publishing.
Seems to me, there are 2 high-level choices:
_1. Core endpoints do not delegate/redirect to plugins._
- POST to /RepositoryVersion/ is removed.
- POST to /Publications/ (stays gone)
- Plugins provide endpoints for sync and other to create new
repository versions.
- Plugins provide endpoints for creating Publications (publishing).
_2. Core delegates behavior to plugins for those endpoints requiring
plugin participation._
- POST to /RepositoryVersion/ is the only way to create a
repository version.
- POST to /Publications/ is the only way to create a Publication
(publish).
- The POST parameters or body includes enough information so that
core can select a plugin.
- Either the entire POST is passed along to the plugin, _or_ the
plugin implements an API that's used by
core for pre-defined participation.
There have been proposals on how both #1 and #2 can be achieved. 'm
wondering if we can even agree on one of these two approaches.
>> A File repository version cannot be properly published if it
>> contains 2 FileContent units that both have the same relative
>> path. The publisher has to decide which FileContent to publish at
>> the relative path. That decision cannot be made intelligently by
>> the publisher. The decision on which content unit to include in
>> the repository version rests with the user that is creating the
>> repository version. When a user tries to create a repository
>> version with a FileContent that has the same relative path as
>> another FileContent that was added previously, Pulp needs to
>> inform the user that there is a conflict (and not create the
>> repositiory version). This validation can only be provided by the
>> File plugin.
>>
>>>
>>> We would like to get feedback on these issues being
>>> sound and worth resolving before we resume
>>> particular solution discussion[1].
>>>
>>> Thanks,
>>> Austin, Dennis, and Milan
>>>
>>> [1]
>>> https://www.redhat.com/archives/pulp-dev/2018-March/msg00066.html
>>> <https://www.redhat.com/archives/pulp-dev/2018-March/msg00066.html>
>>>
>>>
>>> _______________________________________________
>>> Pulp-dev mailing list
>>> Pulp-dev at redhat.com <mailto:Pulp-dev at redhat.com>
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>> <https://www.redhat.com/mailman/listinfo/pulp-dev>
>>>
>>>
>>>
>>> _______________________________________________
>>> Pulp-dev mailing list
>>> Pulp-dev at redhat.com <mailto:Pulp-dev at redhat.com>
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>> <https://www.redhat.com/mailman/listinfo/pulp-dev>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Pulp-dev mailing list
>>> Pulp-dev at redhat.com <mailto:Pulp-dev at redhat.com>
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>> <https://www.redhat.com/mailman/listinfo/pulp-dev>
>>
>>
>> _______________________________________________
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com <mailto:Pulp-dev at redhat.com>
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>> <https://www.redhat.com/mailman/listinfo/pulp-dev>
>>
>>
>
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com <mailto:Pulp-dev at redhat.com>
> https://www.redhat.com/mailman/listinfo/pulp-dev
> <https://www.redhat.com/mailman/listinfo/pulp-dev>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20180413/359e6116/attachment.htm>
More information about the Pulp-dev
mailing list