[Pulp-dev] proposed changes to Pulp 3 auto generated docs
Brian Bouterse
bbouters at redhat.com
Mon Aug 6 16:42:37 UTC 2018
Does anyone have strong feelings or concerns about unnesting repository
versions from /repositories/1/version/1/ to something like /repoversion/1/ ?
As a mini-group of devs talk it over this week, understanding how
users/devs feel about this nesting would be helpful.
Also thank you @bizhang for the great prototyping and communication.
On Fri, Aug 3, 2018 at 4:05 PM, Bihan Zhang <bizhang at redhat.com> wrote:
> I've opened a PR that implements a new field that will resolve and display
> both _href and id.
> https://github.com/pulp/pulp/pull/3575
>
> Take a look and let me know what y'all think.
>
>
> On Thu, Jul 26, 2018 at 2:33 PM, Robin Chan <rchan at redhat.com> wrote:
>
>> I really appreciate recent trend on our mailing lists that when
>> conversations are happening on irc or in meetings, that updates are
>> reported back to the relevant thread. I like that individuals are taking
>> the initiative to have that real time communication and work out concerns
>> in a timely manner, and I really like that everyone else can get caught up.
>> This supports the timezone- diverse pulp community as well as time away for
>> summer travel. I personally have found this update and others like it on
>> other threads here in pulp-land to be very helpful.
>>
>> Thanks,
>> Robin
>>
>> On Thu, Jul 26, 2018 at 12:13 PM, Bihan Zhang <bizhang at redhat.com> wrote:
>>
>>> bmbouter, dkliban, daviddavis, jsherril and I just chatted about this in
>>> person. Here are the minutes from that meeting:
>>>
>>> We have tools (API, docs, bindings+3rd party applications, CLI) that
>>> needs to refer to references somehow, and we need to figure out whether
>>> these references should be the href or a combination of the href and the id.
>>> Dkliban has opened a PR updating the schemas to only use the hrefs, but
>>> this schema isn't valid. [0]
>>> Bmbouter has proposed some solutions to using both hrefs and ids, and
>>> option 2 is by far the most popular [1]
>>>
>>> If we go the hybrid href and id route there are some concerns:
>>> - How do we serialize createdresource that can be either
>>> publications or repositoryversion?
>>> - additionalproperties field in openapi should take care of it
>>> - repository has to be referenced with 2 parameters (repo id and
>>> version number) instead of just an id like all other resources, this isn't
>>> consistent
>>> - maybe we should unnest repoversion, this also allows us to
>>> search for content across repository versoins
>>> - probably should leave as is, since it works, and can be
>>> described in openapiv2
>>> - we need to restructure our API, and this could be time consuming
>>> - changes needed to hyperlinked related field, created resource
>>> schema, content added api endpoint, need to validate openapi schema is
>>> compliant for all requests and responses
>>>
>>> If we merge dkliban's PR, and use only hrefs there's also some concerns:
>>> - noncompliant schema
>>> - we aren't in a position to ship and support the tool chain for
>>> a noncompliant schema, unlike google
>>> - following spec gives some peace of mind for future support
>>>
>>> We decided on the following action items:
>>> - bizhang will take some time to write out a hybrid href and id POC
>>> next week
>>> - dkliban will reach out to openapi, to see what the status is of
>>> the href work, and maybe get an estimation of when that will be accepted
>>> into the specification
>>>
>>>
>>> [0] https://github.com/pulp/pulp/pull/3561#issuecomment-407496845
>>> [1] https://github.com/pulp/pulp/pull/3561#issuecomment-407888652
>>>
>>> On Wed, Jul 25, 2018 at 4:43 PM, Brian Bouterse <bbouters at redhat.com>
>>> wrote:
>>>
>>>> I'm exploring the changes required to use IDs and hrefs on the PR
>>>> here: https://github.com/pulp/pulp/pull/3561#issuecomment-407888652
>>>>
>>>> On Wed, Jul 25, 2018 at 4:24 PM, David Davis <daviddavis at redhat.com>
>>>> wrote:
>>>>
>>>>> I know we don’t support things like accepting hrefs as references to
>>>>> resources but if I remember correctly we do return hrefs alongside ids in
>>>>> responses in Pulp 2. Is that not correct?
>>>>>
>>>>> David
>>>>>
>>>>>
>>>>> On Wed, Jul 25, 2018 at 4:17 PM Dennis Kliban <dkliban at redhat.com>
>>>>> wrote:
>>>>>
>>>>>> I don't think we support both hrefs and ids in Pulp 2. The Pulp 2
>>>>>> REST API does not accept HREFs as references to resources. In Pulp 2's REST
>>>>>> API we do not even have resources that have relationships to other
>>>>>> resources. The relationships between resources are established by nesting
>>>>>> them under one another. e.g.: /pulp/api/v2/repositories/<repo_id>/
>>>>>> and /pulp/api/v2/repositories/<repo_id>/importers/<importer_id>/. In
>>>>>> Pulp 2, if a user wanted to reference content units in a request, the API
>>>>>> requires writing a filter that uses Mongodb syntax.
>>>>>>
>>>>>> Pulp 3's REST API has a resources called Task that has a
>>>>>> 'created_resource' attribute. This resource is a reference to either a
>>>>>> repository version or a publication at this time. Pulp 3's REST API also
>>>>>> supports users specifying references to content units that should be added
>>>>>> or removed from a repository. These needs do not exist in Pulp 2's REST
>>>>>> API.
>>>>>>
>>>>>>
>>>>>> On Wed, Jul 25, 2018 at 3:55 PM, David Davis <daviddavis at redhat.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Correct me if I’m wrong but Pulp 2 supported @bizhang’s model of
>>>>>>> providing both hrefs and ids. Was that a source of problems or complaints
>>>>>>> by Pulp 2 users?
>>>>>>>
>>>>>>> David
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jul 25, 2018 at 3:08 PM Dennis Kliban <dkliban at redhat.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> For everyone following along, the conversation has moved to Github
>>>>>>>> - on the PR[0] with the proposed changes.
>>>>>>>>
>>>>>>>> [0] https://github.com/pulp/pulp/pull/3561
>>>>>>>>
>>>>>>>> On Tue, Jul 24, 2018 at 11:15 AM, Bihan Zhang <bizhang at redhat.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> @dkliban I've tried out your PR and left a question:
>>>>>>>>> https://github.com/pulp/pulp/pull/3561#issuecomment-407425172
>>>>>>>>>
>>>>>>>>> Won't it be problematic with the openapi definitions causing us to
>>>>>>>>>> have two schemas? Accepting the data in two forms is one thing, but using
>>>>>>>>>> openapi to describe it both ways is something I don't understand well. Are
>>>>>>>>>> we going to ship and test two?
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I don't think we'll be defining the data in two different ways in
>>>>>>>>> openapi. We need to pass a {repository identifier} to /sync/, openapi
>>>>>>>>> expects a string, what we do with that string is up to us. (In the
>>>>>>>>> following example the format is "uri" but this isn't actually used for
>>>>>>>>> validation at all, since it's not defined by the swagger specification [0],
>>>>>>>>> we can also clear out the format field, since format is only there to
>>>>>>>>> support documentation needs)
>>>>>>>>>
>>>>>>>>> - RepositorySyncURL:
>>>>>>>>> {
>>>>>>>>> - required:
>>>>>>>>> [
>>>>>>>>> - "repository"
>>>>>>>>> ],
>>>>>>>>> - type: "object",
>>>>>>>>> - properties:
>>>>>>>>> {
>>>>>>>>> - repository:
>>>>>>>>> {
>>>>>>>>> - title: "Repository",
>>>>>>>>> - description: "A URI of the repository to be
>>>>>>>>> synchronized.",
>>>>>>>>> - type: "string",
>>>>>>>>> - format: "uri"
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>> },
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I can see why some users would want to refer to things in the api
>>>>>>>>>> using ID not an href. I think about the case that when calling publish and
>>>>>>>>>> referring to a RepositoryVersion with id=827561, num=3, and for
>>>>>>>>>> repository=1234. With an ID alternately accepted, you could call publish
>>>>>>>>>> and submit repo_version=827561 instead of repo_version='repositories/1234/version/3/'.
>>>>>>>>>> I can see that benefit, but it comes with downsides. Saving/storing a url I
>>>>>>>>>> know feels a little strange, but I do see several upsides...
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Doing it only with hrefs, ensures those benefits (nice recap btw)
>>>>>>>>>> will always be true. Having to submit the references using something like
>>>>>>>>>> 'repositories/1234/version/3/' will cause any client to store them that
>>>>>>>>>> way. I think that's a good thing because someone troubleshooting their
>>>>>>>>>> scripts or in katello's db will instead have 'repositories/1234/version/3/',
>>>>>>>>>> which they can directly use with HTTP. I think this is valuable. Otherwise
>>>>>>>>>> they would have repo version 827561, which now they have to do extra work
>>>>>>>>>> to start interacting with that object via HTTP. Storing urls removes the
>>>>>>>>>> "templating" step from the troubleshooter's responsibilities so we're
>>>>>>>>>> making their job easier. Spacewise, I don't think the clients benefit
>>>>>>>>>> hugely from storing 827561 instead of 'repositories/1234/version/3/',
>>>>>>>>>> but humans do.
>>>>>>>>>>
>>>>>>>>> Why don't we provide the ability to use both href and id as
>>>>>>>>> identifiers, and katello can choose the route that is right for them based
>>>>>>>>> on the points you bring up?
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I don't know much about the CLI, but if we want to enable a
>>>>>>>>>> specific user experience, I think we can find a way to make that work.
>>>>>>>>>> Overall I think users should be able to specify things in the most
>>>>>>>>>> intuitive way possible, and I don't see how API data formats directly
>>>>>>>>>> influence that. For example I think referring to a repository by it's name
>>>>>>>>>> is the most natural; it's more natural than 1234 or repositories/1234.
>>>>>>>>>>
>>>>>>>>> +1 the CLI can resolve name to identifiers (either id or href), so
>>>>>>>>> I'm not too concerned with that.
>>>>>>>>>
>>>>>>>>> [0] https://github.com/OAI/OpenAPI-Specification/blob/master
>>>>>>>>> /versions/2.0.md#data-types
>>>>>>>>>
>>>>>>>>> On Mon, Jul 23, 2018 at 9:51 PM, Dennis Kliban <dkliban at redhat.com
>>>>>>>>> > wrote:
>>>>>>>>>
>>>>>>>>>> I've made a work in progress PR[0] that demonstrates the changes
>>>>>>>>>> I was suggesting.
>>>>>>>>>>
>>>>>>>>>> [0] https://github.com/pulp/pulp/pull/3561
>>>>>>>>>>
>>>>>>>>>> On Mon, Jul 23, 2018 at 3:50 PM, Brian Bouterse <
>>>>>>>>>> bbouters at redhat.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Having two ways to refer to objects in the API makes me nervous.
>>>>>>>>>>> I have some questions/concerns/ideas. I'm also interested to see what
>>>>>>>>>>> dkliban's bindings produce in terms of a resolution of the swagger issues.
>>>>>>>>>>>
>>>>>>>>>>> Won't it be problematic with the openapi definitions causing us
>>>>>>>>>>> to have two schemas? Accepting the data in two forms is one thing, but
>>>>>>>>>>> using openapi to describe it both ways is something I don't understand
>>>>>>>>>>> well. Are we going to ship and test two?
>>>>>>>>>>>
>>>>>>>>>>> I can see why some users would want to refer to things in the
>>>>>>>>>>> api using ID not an href. I think about the case that when calling publish
>>>>>>>>>>> and referring to a RepositoryVersion with id=827561, num=3, and for
>>>>>>>>>>> repository=1234. With an ID alternately accepted, you could call publish
>>>>>>>>>>> and submit repo_version=827561 instead of repo_version='repositories/1234/version/3/'.
>>>>>>>>>>> I can see that benefit, but it comes with downsides. Saving/storing a url I
>>>>>>>>>>> know feels a little strange, but I do see several upsides...
>>>>>>>>>>>
>>>>>>>>>>> Doing it only with hrefs, ensures those benefits (nice recap
>>>>>>>>>>> btw) will always be true. Having to submit the references using something
>>>>>>>>>>> like 'repositories/1234/version/3/' will cause any client to store them
>>>>>>>>>>> that way. I think that's a good thing because someone troubleshooting their
>>>>>>>>>>> scripts or in katello's db will instead have 'repositories/1234/version/3/',
>>>>>>>>>>> which they can directly use with HTTP. I think this is valuable. Otherwise
>>>>>>>>>>> they would have repo version 827561, which now they have to do extra work
>>>>>>>>>>> to start interacting with that object via HTTP. Storing urls removes the
>>>>>>>>>>> "templating" step from the troubleshooter's responsibilities so we're
>>>>>>>>>>> making their job easier. Spacewise, I don't think the clients benefit
>>>>>>>>>>> hugely from storing 827561 instead of 'repositories/1234/version/3/',
>>>>>>>>>>> but humans do.
>>>>>>>>>>>
>>>>>>>>>>> I don't know much about the CLI, but if we want to enable a
>>>>>>>>>>> specific user experience, I think we can find a way to make that work.
>>>>>>>>>>> Overall I think users should be able to specify things in the most
>>>>>>>>>>> intuitive way possible, and I don't see how API data formats directly
>>>>>>>>>>> influence that. For example I think referring to a repository by it's name
>>>>>>>>>>> is the most natural; it's more natural than 1234 or repositories/1234.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jul 19, 2018 at 8:30 AM, Daniel Alley <dalley at redhat.com
>>>>>>>>>>> > wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Keep in mind that as of yesterday, unless we revert the change,
>>>>>>>>>>>> we are using Integers IDs instead of UUIDs
>>>>>>>>>>>>
>>>>>>>>>>>> https://github.com/pulp/pulp/pull/3549
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jul 18, 2018 at 9:57 PM, Bihan Zhang <
>>>>>>>>>>>> bizhang at redhat.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jul 18, 2018 at 1:05 PM, Dennis Kliban <
>>>>>>>>>>>>> dkliban at redhat.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I was asked on IRC to state what problems the proposed
>>>>>>>>>>>>>> changes are trying to address. There are two problems I see with the
>>>>>>>>>>>>>> current OpenAPI 2.0 schema Pulp's REST API provides.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - The path parameters in the schema don't reflect the
>>>>>>>>>>>>>> parameters our users should be using for identifying the resources
>>>>>>>>>>>>>> available via REST API.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm not convinced that we should use hrefs as the sole
>>>>>>>>>>>>> identifiers for the resources.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Here are the reasons I see that we use hrefs as identifiers in
>>>>>>>>>>>>> a REST API context:
>>>>>>>>>>>>> 1. Each href provides full context into the resource it
>>>>>>>>>>>>> identifies. When given a href you would know exactly which resource it is
>>>>>>>>>>>>> referencing and would never run into the issue of: what is this {uuid}
>>>>>>>>>>>>> because you know it is a 'repositories/{uuid}'
>>>>>>>>>>>>> 2. discoverability, you know exactly how to access
>>>>>>>>>>>>> resources from hitting the root url (and in a webui can just click)
>>>>>>>>>>>>> 3. You would not need to construct urls from templates
>>>>>>>>>>>>>
>>>>>>>>>>>>> But things are different if we look at it from a
>>>>>>>>>>>>> bindings/client context. The difference is mainly due to how
>>>>>>>>>>>>> discoverability is done: in the REST API context the user has little prior
>>>>>>>>>>>>> knowledge to what resources are available, and how to access theses
>>>>>>>>>>>>> resoruces. But the bindings/client are generated from the schema, which
>>>>>>>>>>>>> defines exactly how resources are structured, and what the context of each
>>>>>>>>>>>>> {uuid} is.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1. Given an {uuid} the client/bindings knows exactly what
>>>>>>>>>>>>> resource this {uuid} refers to. With hrefs there would be redundant
>>>>>>>>>>>>> information pulp.repositories('repositories/{uuid}') (why do
>>>>>>>>>>>>> I need to specify repositories twice?)
>>>>>>>>>>>>> 2. Discoverability is done with the schema which contains
>>>>>>>>>>>>> all the information about available resources/endpoints
>>>>>>>>>>>>> 3. URL construction is done by the client, so the user
>>>>>>>>>>>>> would also never need to do any url construction themselves (unless we
>>>>>>>>>>>>> continue to force href only identifiers, in which case they might have to
>>>>>>>>>>>>> do some url construction to pass as arguments)
>>>>>>>>>>>>>
>>>>>>>>>>>>> I don't think hrefs and uuid identifiers are mutually
>>>>>>>>>>>>> exclusive. I propose that we extend HyperlinkedRelatedFields to accept
>>>>>>>>>>>>> either href or uuid, and map these HyperlinkedRelatedFields to each other
>>>>>>>>>>>>> in the schema with openapi definition objects [0].
>>>>>>>>>>>>>
>>>>>>>>>>>>> [0] https://github.com/OAI/OpenAPI
>>>>>>>>>>>>> -Specification/blob/master/versions/2.0.md#responses-definit
>>>>>>>>>>>>> ions-object
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - The path parameters don't have a description in the schema.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1 to updating the schema descriptions for these parameters
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Do others agree with these problem statements?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jul 18, 2018 at 9:31 AM, Dennis Kliban <
>>>>>>>>>>>>>> dkliban at redhat.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I am working on improving the OpenAPI 2.0 schema for Pulp 3.
>>>>>>>>>>>>>>> I would like to get some input on the improvements I am proposing. The
>>>>>>>>>>>>>>> schema is used to generate our REST API documentation as well as the
>>>>>>>>>>>>>>> bindings with swagger-codegen.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The docs generated from our current schema look something
>>>>>>>>>>>>>>> like this:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> GET /repositories/{repository_pk}/versions/{number}/content/
>>>>>>>>>>>>>>> <https://docs.pulpproject.org/en/3.0/nightly/integration-guide/rest-api/index.html#get--repositories-repository_pk-versions-number-content->
>>>>>>>>>>>>>>> Parameters:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - *number* (*integer*) –
>>>>>>>>>>>>>>> - *repository_pk* (*string*) –
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Status Codes:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - 200 OK
>>>>>>>>>>>>>>> <http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.2.1>
>>>>>>>>>>>>>>> –
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Since Pulp identifies resources using their HREFs, I am
>>>>>>>>>>>>>>> proposing that the schema produce documentation that states:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> GET /{repository_version_href}/content/
>>>>>>>>>>>>>>> Parameters:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - *repository_version_href* (string) – HREF for the
>>>>>>>>>>>>>>> repository version
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Status Codes:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - 200 OK
>>>>>>>>>>>>>>> <http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.2.1>
>>>>>>>>>>>>>>> –
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thoughts? Ideas? All feedback is welcome. Thank you!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> Pulp-dev mailing list
>>>>>>>>>>>>>> Pulp-dev at redhat.com
>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Pulp-dev mailing list
>>>>>>>>>>>>> Pulp-dev at redhat.com
>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Pulp-dev mailing list
>>>>>>>>>>>> Pulp-dev at redhat.com
>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Pulp-dev mailing list
>>>>>>>>>>> Pulp-dev at redhat.com
>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Pulp-dev mailing list
>>>>>>>>>> Pulp-dev at redhat.com
>>>>>>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Pulp-dev mailing list
>>>>>>>> Pulp-dev at redhat.com
>>>>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>>>>
>>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> Pulp-dev mailing list
>>>>> Pulp-dev at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Pulp-dev mailing list
>>>> Pulp-dev at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Pulp-dev mailing list
>>> Pulp-dev at redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20180806/e0358dba/attachment.htm>
More information about the Pulp-dev
mailing list