<div dir="ltr"><div>To make sure this was feasable, I've created proof of concept PRs for pulpcore and pulp_file. All of the proposed changes are proofed, but not all work is done. For example, sync is implemented, but not publish.</div><div><ul><li><a href="https://github.com/pulp/pulp/pull/3394">https://github.com/pulp/pulp/pull/3394</a> </li><li><a href="https://github.com/pulp/pulp_file/pull/61">https://github.com/pulp/pulp_file/pull/61</a> </li></ul><div>I am very pleased with how it turned out, and I think it will make plugin writing even easier. I documented the design on a redmine issue. I encourage feedback/discussion on the issue.</div></div><div> </div><div><a href="https://pulp.plan.io/issues/3522">https://pulp.plan.io/issues/3522</a> </div></div><div class="gmail_extra"> <div class="gmail_quote">On Tue, Mar 20, 2018 at 7:46 AM, Milan Kovacik <<a href="mailto:mkovacik@redhat.com" target="_blank">mkovacik@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Fri, Mar 16, 2018 at 4:55 PM, Milan Kovacik <<a href="mailto:mkovacik@redhat.com">mkovacik@redhat.com</a>> wrote: > > > > On Thu, Mar 15, 2018 at 9:21 PM, Austin Macdonald <<a href="mailto:austin@redhat.com">austin@redhat.com</a>> wrote: >> >> I spoke with daviddavis about this and I would like to narrow the scope a bit. This discussion should be limited to endpoints that deploy tasks. The possibility for collisions that David pointed out regarding v3/content/<type>/ is real, but should be discussed separately because Content objects should follow the pattern of the other plugin defined objects. All of them are master/detail so the namespacing v3/plugin/<type>/content/ would go deeply against the grain. >> >> >> On Mar 15, 2018 10:53, "David Davis" <<a href="mailto:daviddavis@redhat.com">daviddavis@redhat.com</a>> wrote: >>> >>> I have an amendment to this proposal. Instead of namespacing just the plugin task routes, I’d propose we namespace all plugin routes. Thus all plugin routes would be namespaced under something like “/api/v3/plugin/<plugin name>/“. For example, “/api/v3/content/file/” gets moved to “/api/v3/plugin/file/content/” and so on. >>> >>> >>> For task routes, plugins could still route to a pulpcore viewset/action: >>> >>> User route: POST /api/v3/plugin/file/repository_version/ add_content=[…] >>> >>> Routes to: POST /api/v3/tasks plugin=file task=create_repository_version add_content=[…] >> >> >> AFAIK, we don't have a good mechanism for validating and re-routing endpoints quite like this. Instead, a view or viewset would be created, validation performed, and then a task is deployed. The task itself could be a vanilla task from pulpcore though. Another issue is that a POST to v3/plugin/file/repository_version/ implies that the resultant repository_version is typed in some way. The created resource href would still take the form v3/repositories/1234/versions/2/, so I think this is a little misleading. > > > +1 to generic Pulp concepts/objects staying outside of a plug-in namespace/path > >> >> >> For the other endpoint: >> /api/v3/tasks plugin=file task=create_repository_version add_content=[…] >> >> There is a still the problem is that POST requests should not contain parameters that influence the allowable parameter set. For instance, plugin=python might require an additional parameter that is not allowed for the file plugin. In particular, this affects the auto-documented REST API. >> >> >> However, the concept in general could work. I've adapted David's suggestion and will present it side by side with the original idea. Note, this only applies to "actions", which are sync, publish, add/remove, and plugin-specific actions (including rich-dependency adding). >> >> The difference between the two ideas is based in semantics. Both would deploy the same task functions. >> >> "Action Based" >> POST v3/plugin/file/sync/ >> Note that "sync" is singular. This is a file-plugin action to sync using the body parameters. > > > - 1 as this might imply mixing <synchronous> and <asynchronous> actions while all the time, asynchronous actions, such as the sync call, always create a task object; the user would be creating the task objects at one endpoint but deleting, filtering, tracking those at another. > >> >> >> "Task Object Based" >> POST v3/tasks/file/syncs/ >> Note that "syncs" is plural. This is a POST that creates a file-sync task object,. Btw an interesting consequence of tasks not being objects in a file importer and publisher are these two issues: - The parameters in the API schema for the sync and publish endpoints are incorrect [1] - Not able to access documentation endpoint [2] In the former issue, the DRF isn't able to automatically figure out schema for e.g the `sync` endpoint of the file plug-in importer. The reason for it is even though the `sync` verb is attached to an importer object, the parameters passed in aren't supposed to be importer details so they shouldn't use the importer serializer to get e.g auto-documented. Moreover, the result of a call to this endpoint is supposed to be a task (json object), schema and serialization of which has to be repeated for the `sync` endpoint to some extent. Last but not least, it isn't clear whether the `sync` verb should belong to an importer or a repo, or something else as the call "responsibility" is spread somewhat between both and none. The latter issue is a consequence of the lack of support in DRF for what the `sync` verb requires documenting thru its schema, which IMO underlines the `sync` endpoint being odd, a misplaced task probably. Cheers, milan [1] <a href="https://pulp.plan.io/issues/3351" rel="noreferrer" target="_blank">https://pulp.plan.io/issues/3351</a> [2] <a href="https://pulp.plan.io/issues/3420" rel="noreferrer" target="_blank">https://pulp.plan.io/issues/3420</a> <div class="HOEnZb"><div class="h5"> > > + 1 for typed objects > > # adding content to a repo version (already mentioned by Austin but I like it) > POST@v3/tasks/docker/content_additions/ > > # synchronize the Zoo repo > pulp-admin create tasks rpm syncs --repo-id zoo --importer-id zoo-importer --sync-config-override '{"ssl": "false"}' --format=json | tee /tmp/inttest01_sync_task.json | pulp-admin track tasks --format json - > > # get all docker content additions tasks since yesterday > pulp-admin list tasks docker content_additions --format=csv --fields=id,worker_id --filter=started_at__gt=<yesterday> > > # get all tasks since yesterday > pulp-admin list tasks --filter=started_at__gt=<yesterday> --format=yaml > >> >> Discussion: >> Another consideration is "Where will a Live API live?". A benefit of David's proposal is that we would be providing plugins with a namespaced area to do whatever they need, from live api to actions. We probably would want to create this namespace for things like live apis even if we go in the tasks/file/syncs/ direction. The only downside of this part of David's proposal is that our endpoints will still be "action" verbs instead of nouns, and I am ok with that. > > > I'd suggest live API actions that create task object subclasses (generic Pulp concept) reside in namespaced tasks path; this would still allow the user to apply generic concepts > POST@/v3/tasks/file/frobnications/ > > pulp-admin create tasks file frobications --foo=bar --format json | pulp-admin track tasks --format json -- > > Other live API endpoints (not creating task objects) might be kept under namespaces > POST@v3/plugins/file/shake/<{level: strong} > > pulp-admin plugins file shake --level=strong > >> >> _______________________________________________ >> Pulp-dev mailing list >> <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> >> <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> >> > > Cheers, > milan > </div></div></blockquote></div> </div>