[Pulp-dev] Pulp api seemingly incompatible with generated bindings

Dennis Kliban dkliban at redhat.com
Wed Jul 18 13:10:21 UTC 2018 

I have re-evaluated the approach for solving the problem with our bindings.
I have stopped all work on modifying swagger-codegen. swagger-codegen does
a great job of generating bindings from an OpenAPI 2.0 schema definition.
The source of our problems is in the schema itself. I have decided to focus
on improving our schema definition to better reflect how we want to
describe the REST API.

On Wed, Jul 11, 2018 at 4:03 PM, Dennis Kliban <dkliban at redhat.com> wrote:

> On Mon, Jul 9, 2018 at 9:16 PM, Justin Sherrill <jsherril at redhat.com>
> wrote:
>
>> Hi all,
>>
>> I finally had some time to play with these and I do have some concerns.
>> First i'd say that Dennis' work to get the swagger generation working with
>> href's seemed to work really well!  I did find one small bug(1) that may be
>> isolated to the ruby bindings, but seems like a serialization issue
>> potentially.
>>
> Thank you for filing the issue. We are going to add it to our next sprint
> on Friday.
>
>
>> However I do have some concerns.  According to Dennis, upstream swagger
>> does not seem open to accepting this change that utilizes href's in its api
>> calls.  This leaves us all in a bit of a pickle as:
>>
>> 1) pulp seems to suggest to use hrefs (and may remove ids in the future?)
>> 2) swagger does not (and will never?) support referencing objects by
>> href's in the future
>>
> I would like to clear up the language we are using.
>
> swagger[0] is the former name for OpenAPI 2.0, which is a standard for
> richly defining REST API schema.
>
> Pulp 3 has a REST API endpoint that serves it's OpenAPI 2.0 schema. We are
> constantly working to improve the accuracy and level of detail of this
> OpenAPI 2.0 schema. We use the schema to generate our REST API
> documentation. Our CLI is going to rely on the schema to inform the user
> what all interactions with the server are available and what parameters are
> needed. We have recently decided to start using pyswagger[1] as the client
> library in the CLI. I have not been able to find a similar client for Ruby.
> I welcome others to search for one.
>
> Another solution for taking advantage of the OpenAPI 2.0 schema definition
> on the client side is to generate a client using swagger-codegen[2].
> swagger-codegen is a tool that takes OpenAPI 2.0 schema definition as an
> input and produces a client library (bindings) in a language of your
> choice. It does so by taking parameters it finds in the schema and inserts
> them into templates for the specific language requested. The templates
> bundled with swagger-codegen assume that the user will use "path
> parameters" to identify resources. This behavior is not compatible with
> Pulp because Pulp identifies resources using URIs.
>
> I asked in #swagger on freenode about changing the behavior of
> swagger-codegen and I was recommended to update the templates for Ruby,
> Python, and any other languages we want to support. That is how users of
> swagger-codegen are supposed to customize the behavior of swagger-codegen.
>
> Instead of modifying each language's template, I found it easier to modify
> the behavior of swagger-codegen. As a result, we can generate working
> bindings in all the languages supported by swagger-codegen.
>
> [0] https://github.com/OAI/OpenAPI-Specification/blob/master/ver
> sions/2.0.md
> [1] https://github.com/pyopenapi/pyswagger
> [2] https://swagger.io/tools/swagger-codegen/
>
>
>> Thus for katello to use href's, we would need to use an altered version
>> of swagger that someone maintains.  While the patch is small, I'm not sure
>> either of our teams are super well positioned to maintain something like
>> this long term.
>>
> I based my changes on the master branch, which will be the next, 2.4.0,
> release of swagger-codegen. I have also checked that a similar patch can be
> used for modifying the behavior on the 3.0.0 branch. We should not have
> compatibility problems with swagger-codegen 2.y and a similar approach can
> be used with 3.y when that becomes the stable release.
>
>
>> It might make sense for the pulp team to maintain this, as the
>> requirements are being suggested by the pulp team, however it doesn't seem
>> like ya'll will actually be using any generated bindings which doesn't seem
>> like a good dependency.
>>
> We want to make the modified version of swagger-codegen available to all
> Pulp users. Our users will then have an option of using an unpatched
> version of swagger-codegen to generate bindings that require the user to
> remember both UUIDs and HREFs or using our version of swagger-codegen that
> will produce bindings that use URI as the identifier in all cases.
>
>
>> It might make sense for the Katello team but I would say we're only using
>> hrefs because they seem to be what was recommended.  In both cases its a
>> different technology (java) with a different built system.  I could see 3
>> years from now some issue coming up that requires us to update to a newer
>> swagger version and untangle this when no one that was involved in this
>> decision was around.  I don't really see either of our teams maintaining a
>> fork of the swagger code generation as a good long term solution.
>>
>
> The Pulp team will have ownership of the modified swagger-codegen as it
> will be providing value to all users that want to integrate with Pulp 3.
>
> We do not intend to publish bindings for any particular language because
> each Pulp installation will require a different set of bindings based on
> which plugins are installed.
>
>
>> May I suggest that we either:
>>
>>  a) Work harder to get the code change upstream
>>
>> or if that does not seem to be going well
>>
>>  b) commit to continue responding with ids (in addition to hrefs) in all
>> object responses
>>
>> (1)  https://pulp.plan.io/issues/3843
>>
> I don't think we plan to stop responding with IDs. However, our REST API
> will continue to expect HREFs in body parameters that reference resources.
> This leaves you with 3 options:
>
>  - Write your own client for interacting with Pulp 3
>
>  - use the upstream version of swagger-codegen to generate bindings and
> keep track of IDs and HREFs
>
>  - use the version of swagger-codegen shipped by the Pulp team and only
> keep track of HREFs
>
>
>
>>
>>
>> On 05/17/2018 06:26 PM, Dennis Kliban wrote:
>>
>> I have been able modify the default behavior of swagger-codegen to
>> produce bindings that use relative URLs to identify resources. The required
>> changes can be seen here[0]. The compiled jar is available for download
>> here[1].
>>
>> The following command can be used to generate python bindings. Replace
>> 'python' with 'ruby' and you'll get ruby bindings. The json file is the API
>> schema returned by the Pulp server at http://localhost:8000/api/v3/d
>> ocs/api.json.
>>
>> java -jar ./swagger-codegen-cli.jar generate -i
>> ~/Downloads/pulp3-api.json -l python -o some_directory_name
>>
>> [0] https://github.com/dkliban/swagger-codegen/commit/e31e967698
>> 64b73b06bd99f5e20e4720562539b9
>> [1] https://repos.fedorapeople.org/pulp/pulp/swagger/swagger-cod
>> egen-cli.jar
>>
>> On Tue, May 1, 2018 at 11:18 AM, David Davis <daviddavis at redhat.com>
>> wrote:
>>
>>> We’ve exposed IDs and I think the next goal will be to somehow let
>>> Katello use these IDs. I’ve opened a new story to solicit feedback on how
>>> this’ll work:
>>>
>>> https://pulp.plan.io/issues/3636
>>>
>>>
>>> David
>>>
>>> On Tue, May 1, 2018 at 9:14 AM, Brian Bouterse <bbouters at redhat.com>
>>> wrote:
>>>
>>>>
>>>> On Tue, May 1, 2018 at 7:59 AM, Bryan Kearney <bkearney at redhat.com>
>>>> wrote:
>>>>
>>>>> On 04/30/2018 04:08 PM, Brian Bouterse wrote:
>>>>> > @asmacdo, checking in on the why is great. I want to try to
>>>>> articulate
>>>>> > the benefits as I see them. Other perspectives and discussion are
>>>>> welcome.
>>>>> >
>>>>> > The design of using URLs for referring things is a design whose goal
>>>>> is
>>>>> > to minimize complexity as the # of resources grows. The Internet is a
>>>>> > useful analogy here. When someone wants to tell me how to find
>>>>> something
>>>>> > on Instagram, if the article's name is 'cat_pic432642' and that's
>>>>> all I
>>>>> > know, I'm going to have a hard time figuring out the actual URL to
>>>>> ask
>>>>> > Instagram's servers for, e.g. /thepostsarehere/cat_pic432642. Even
>>>>> if I
>>>>> > did know how Instram's url space was laid out, I have to think about
>>>>> it
>>>>> > different from all other web services I interact with; now they're
>>>>> all
>>>>> > different. This is the power of the URL itself. All clients and all
>>>>> > servers can use the uniform resource locator to refer to things. I
>>>>> think
>>>>> > this was the main contribution from Tim Berners-Lee that allowed him
>>>>> to
>>>>> > implement HTTP which has URIs at its heart.
>>>>>
>>>>> But, to Austins Point, if folks are going to interact with Pulp either
>>>>> from the cli or from Go|rust|python libraries, they will not see the
>>>>> urls. In your analogy, you would most likely google cat_pic432642  and
>>>>> never know the url you are going to.
>>>>>
>>>>
>>>> I agree there will be much binding-based usage, and CLI usage, but
>>>> there will always be users who will use httpie with some bash scripting
>>>> calls instead. So from that perspective, the goals remain the same they
>>>> have for years; we need to put forth the best, lowest-complexity REST API.
>>>>
>>>>
>>>>>
>>>>> -- bk
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Pulp-dev mailing list
>>>> Pulp-dev at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Pulp-dev mailing list
>>> Pulp-dev at redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>>
>>
>>
>> _______________________________________________
>> Pulp-dev mailing listPulp-dev at redhat.comhttps://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>>
>> _______________________________________________
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20180718/34c744b0/attachment.htm>

More information about the Pulp-dev mailing list