[Pulp-dev] Solving the "callback problem" ... aka how pulpcore will stop finalizing RepositoryVersion

[Brian Bouterse]

On Tue, Nov 5, 2019 at 3:53 PM Simon Baatz <gmbnomis at gmail.com> wrote:

> Hi Brian,
>
> On Tue, Nov 05, 2019 at 02:46:18PM -0500, Brian Bouterse wrote:
> >    ...
> >    Note that the context manager is only syntactic sugar. The pulp_file
> >    sync code could also just as easily be as shown below. This is
> >    incomplete, but I think you'll get the idea.
> >    [3]
> https://github.com/dralley/pulp_file/compare/typed-repositories...bm
> >    bouter:plugin-finalize-no-context-manager
> >    With this plugins can even do what they want in terms of style
> (context
> >    manager or not). Also they can not use it at all and the only extra
> >    responsibility would be to finalize the RepositoryVersion with its
> >    context manager (core provided).
> >    I'd like to ask for feedback on this design asap. Questions are
> >    concerns ... please send 'em.
>
> This is an elegant solution for the sync case and, as you said,
> offers multiple options to implement validation/extra changes.
> However there are other places in core that create repo versions.  I
> think we have three overall: sync, add_and_remove task, and create()
> in SingleArtifactContentUploadSerializer.
>
> How will the plugin writer reuse validation code in the other places?
> Currently, it seems that he/she has to duplicate some of the existing
> core code to get the plugin's validation code into the right place.
>

Thank you for bringing this up!

Here's one idea I had for the uploads use case:
https://gist.github.com/bmbouter/81be284e6296c541ef9c4a0efc9b304f
Note the it's not the same pattern because core's code is still performing
the finalization, but the plugin writer is able to pass in a fully
configured context manager so in that sense it's not a "hook".

For the add_and_remove task, this change would be on top of typed repos.
add_and_remove is called from only 1 place in @dralley's PR here:
https://github.com/pulp/pulpcore/pull/354/files#diff-03cc41a4bcada47c767bb1615dc53cefR130
For that case we could decompose the 'modify' such that it could be used
as-is if the plugin writer does not require the extra opportunity. If they
do, then they could instantiate the plugin_context_manager in their viewset
and pass it along to the task which could be modified like this:

https://gist.github.com/bmbouter/0897648ce254faecbea730159f4e9cd1

Another option is to move @modify from pulpcore to the plugins themselves,
but then plugins have to opt-in instead of opt-out.

I expect instantiating the context manager in the viewset and passing the
instance to the backend will work because RQ pickles it's job params:
http://python-rq.org/docs/#on-the-design

Note these gists aren't very "clean" but they should illustrate the idea.
How about these approaches?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20191105/e2516a23/attachment.htm>