[Pulp-dev] problems with publishers in Pulp 3

[Brian Bouterse]

After some discussion the proposal has been adjusted to leave publishers
as-is and only introduce Master/Detail Publications with this change.
https://pulp.plan.io/issues/4678#note-6

Please provide more feedback as this likely will be work we try to include
for rc2.

On Thu, Apr 11, 2019 at 4:20 PM Brian Bouterse <bbouters at redhat.com> wrote:

> I wrote up the story on switching publications to Master/Detail and
> removing publishers.  https://pulp.plan.io/issues/4678
>
> Please read it through and ask questions. I believe adding it can be done
> in a backwards compatible way so it's addition in maybe rc2 shouldn't be
> disruptive. The disruptive part will be if Publishers get removed after
> that (see last paragraph on ticket), but we could plan that removal for rc3
> perhaps and mark Publisher codepaths as deprecated in rc2.
>
> On Thu, Apr 11, 2019 at 10:38 AM Dennis Kliban <dkliban at redhat.com> wrote:
>
>> The docker plugin is going to get rid of Publishers and Publications[0].
>> We are planning to update the DockerDistribution with a 'RepositoryVersion'
>> field. If any other plugins want to do the same thing, we will need to make
>> a change in pulpcore to the Distribution model.
>>
>> [0] https://pulp.plan.io/issues/4669
>>
>> On Mon, Apr 8, 2019 at 4:44 PM Brian Bouterse <bbouters at redhat.com>
>> wrote:
>>
>>> Thank you for writing this out. The most significant issue I read in
>>> this is that 3 of the 9 plugins are having their users take steps that
>>> aren't adding any value in their workflow. They want to (and have an
>>> opportunity to) take repository version content and expose it directly.
>>> They don't need Publications or Publishers. If they need no metadata,
>>> instead of a "pass-through" publication it would be easier to have a
>>> Distribution refer to a RepositoryVersion directly and remove the
>>> "pass-through" feature. They the user could skip this step (
>>> https://github.com/pulp/pulp_ansible#create-a-publication ) and instead
>>> post the RepositoryVersion reference as part of the Distributor itself (
>>> https://github.com/pulp/pulp_ansible#create-a-distribution-for-the-publication
>>> ).
>>>
>>> The second issue I read here is that having users CRUD a publisher and
>>> then call that publisher is not as helpful/useful as CRUDing the resulting
>>> Publication directly. One practical issue related to this is the saving of
>>> the "publish" parameters. How do we store those over time? Our modeling
>>> choices have made this a bit challenging because Publication's aren't
>>> plugin controlled. A simple remedy is to have plugins provide various
>>> Publication objects instead of Publishers. These Publications would be
>>> managed by Master/Detail and when created it would run the task that
>>> creates the Publication and return the 202. This simplifies Pulp in that
>>> the options that were used to create the publication can be saved on the
>>> Publication as provided by various plugins. It also allows users to take
>>> one less step (they don't need to make a publisher to then make a
>>> publication). Instead users that need metadata generated can do so with one
>>> step, making the publication. So my main question is, can anyone remember
>>> why we didn't use Master/Detail in this area originally?
>>>
>>> I believe these are changes we should explore. The top one is a pretty
>>> simple add on and mostly not a breaking change. The bottom one would be a
>>> larger change, but one that I believe we could make and should seriously
>>> consider pre 3.0 GA.
>>>
>>>
>>>
>>> On Mon, Apr 8, 2019 at 7:42 AM Dennis Kliban <dkliban at redhat.com> wrote:
>>>
>>>> TLDR:
>>>>
>>>> Auto-distribution of publications is performed implicitly instead of
>>>> explicitly.
>>>> Plugins that don't generate metadata during publish have to provide a
>>>> generic publisher.
>>>> Users have to keep track of publishers to make sure auto-distribution
>>>> of new publications works.
>>>>
>>>> More background:
>>>>
>>>> Publishers in Pulp 3 serve three distinct functions:
>>>>   - store configuration for how a publication should be created
>>>>   - create publications using the configuration
>>>>   - update any distributions that are supposed to be auto updated with
>>>> new publications of a repository (auto distribution).
>>>>
>>>> Ansible, Docker, and Maven plugins do not generate any metadata when
>>>> creating a publication. So their publishers don't need to store any
>>>> configuration. Users of these plugins only get two benefits from publishers:
>>>>   - create publications using the configuration
>>>>   - update any distributions that are supposed to be auto updated with
>>>> new publications of a repository (auto distribution).
>>>>
>>>> The publications created by the publishers of these plugins could be
>>>> created using a single generic publish task. So the only remaining benefit
>>>> of a publisher for these plugins is the auto-distiribution that occurs when
>>>> a Distribution has a repository and a pubisher configured. The only way
>>>> this feature is documented right now is with the help text on the
>>>> 'repository' and 'publisher' fields of Distributions.
>>>>
>>>> Does anyone else see these as problems with the Publishers?
>>>> _______________________________________________
>>>> Pulp-dev mailing list
>>>> Pulp-dev at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20190412/c6a3cebc/attachment.htm>