[Pulp-dev] problems with publishers in Pulp 3

[Brian Bouterse]

I wrote up the story on switching publications to Master/Detail and
removing publishers.  https://pulp.plan.io/issues/4678

Please read it through and ask questions. I believe adding it can be done
in a backwards compatible way so it's addition in maybe rc2 shouldn't be
disruptive. The disruptive part will be if Publishers get removed after
that (see last paragraph on ticket), but we could plan that removal for rc3
perhaps and mark Publisher codepaths as deprecated in rc2.

On Thu, Apr 11, 2019 at 10:38 AM Dennis Kliban <dkliban at redhat.com> wrote:

> The docker plugin is going to get rid of Publishers and Publications[0].
> We are planning to update the DockerDistribution with a 'RepositoryVersion'
> field. If any other plugins want to do the same thing, we will need to make
> a change in pulpcore to the Distribution model.
>
> [0] https://pulp.plan.io/issues/4669
>
> On Mon, Apr 8, 2019 at 4:44 PM Brian Bouterse <bbouters at redhat.com> wrote:
>
>> Thank you for writing this out. The most significant issue I read in this
>> is that 3 of the 9 plugins are having their users take steps that aren't
>> adding any value in their workflow. They want to (and have an opportunity
>> to) take repository version content and expose it directly. They don't need
>> Publications or Publishers. If they need no metadata, instead of a
>> "pass-through" publication it would be easier to have a Distribution refer
>> to a RepositoryVersion directly and remove the "pass-through" feature. They
>> the user could skip this step (
>> https://github.com/pulp/pulp_ansible#create-a-publication ) and instead
>> post the RepositoryVersion reference as part of the Distributor itself (
>> https://github.com/pulp/pulp_ansible#create-a-distribution-for-the-publication
>> ).
>>
>> The second issue I read here is that having users CRUD a publisher and
>> then call that publisher is not as helpful/useful as CRUDing the resulting
>> Publication directly. One practical issue related to this is the saving of
>> the "publish" parameters. How do we store those over time? Our modeling
>> choices have made this a bit challenging because Publication's aren't
>> plugin controlled. A simple remedy is to have plugins provide various
>> Publication objects instead of Publishers. These Publications would be
>> managed by Master/Detail and when created it would run the task that
>> creates the Publication and return the 202. This simplifies Pulp in that
>> the options that were used to create the publication can be saved on the
>> Publication as provided by various plugins. It also allows users to take
>> one less step (they don't need to make a publisher to then make a
>> publication). Instead users that need metadata generated can do so with one
>> step, making the publication. So my main question is, can anyone remember
>> why we didn't use Master/Detail in this area originally?
>>
>> I believe these are changes we should explore. The top one is a pretty
>> simple add on and mostly not a breaking change. The bottom one would be a
>> larger change, but one that I believe we could make and should seriously
>> consider pre 3.0 GA.
>>
>>
>>
>> On Mon, Apr 8, 2019 at 7:42 AM Dennis Kliban <dkliban at redhat.com> wrote:
>>
>>> TLDR:
>>>
>>> Auto-distribution of publications is performed implicitly instead of
>>> explicitly.
>>> Plugins that don't generate metadata during publish have to provide a
>>> generic publisher.
>>> Users have to keep track of publishers to make sure auto-distribution of
>>> new publications works.
>>>
>>> More background:
>>>
>>> Publishers in Pulp 3 serve three distinct functions:
>>>   - store configuration for how a publication should be created
>>>   - create publications using the configuration
>>>   - update any distributions that are supposed to be auto updated with
>>> new publications of a repository (auto distribution).
>>>
>>> Ansible, Docker, and Maven plugins do not generate any metadata when
>>> creating a publication. So their publishers don't need to store any
>>> configuration. Users of these plugins only get two benefits from publishers:
>>>   - create publications using the configuration
>>>   - update any distributions that are supposed to be auto updated with
>>> new publications of a repository (auto distribution).
>>>
>>> The publications created by the publishers of these plugins could be
>>> created using a single generic publish task. So the only remaining benefit
>>> of a publisher for these plugins is the auto-distiribution that occurs when
>>> a Distribution has a repository and a pubisher configured. The only way
>>> this feature is documented right now is with the help text on the
>>> 'repository' and 'publisher' fields of Distributions.
>>>
>>> Does anyone else see these as problems with the Publishers?
>>> _______________________________________________
>>> Pulp-dev mailing list
>>> Pulp-dev at redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20190411/c3ff458c/attachment.htm>