[Pulp-dev] problems with publishers in Pulp 3

[Ina Panova]

I updated the description of the story and removed the "Publishers Removal"
part.

--------
Regards,

Ina Panova
Software Engineer| Pulp| Red Hat Inc.

"Do not go where the path may lead,
 go instead where there is no path and leave a trail."


On Fri, Apr 12, 2019 at 10:53 PM Brian Bouterse <bbouters at redhat.com> wrote:

> After some discussion the proposal has been adjusted to leave publishers
> as-is and only introduce Master/Detail Publications with this change.
> https://pulp.plan.io/issues/4678#note-6
>
> Please provide more feedback as this likely will be work we try to include
> for rc2.
>
> On Thu, Apr 11, 2019 at 4:20 PM Brian Bouterse <bbouters at redhat.com>
> wrote:
>
>> I wrote up the story on switching publications to Master/Detail and
>> removing publishers.  https://pulp.plan.io/issues/4678
>>
>> Please read it through and ask questions. I believe adding it can be done
>> in a backwards compatible way so it's addition in maybe rc2 shouldn't be
>> disruptive. The disruptive part will be if Publishers get removed after
>> that (see last paragraph on ticket), but we could plan that removal for rc3
>> perhaps and mark Publisher codepaths as deprecated in rc2.
>>
>> On Thu, Apr 11, 2019 at 10:38 AM Dennis Kliban <dkliban at redhat.com>
>> wrote:
>>
>>> The docker plugin is going to get rid of Publishers and Publications[0].
>>> We are planning to update the DockerDistribution with a 'RepositoryVersion'
>>> field. If any other plugins want to do the same thing, we will need to make
>>> a change in pulpcore to the Distribution model.
>>>
>>> [0] https://pulp.plan.io/issues/4669
>>>
>>> On Mon, Apr 8, 2019 at 4:44 PM Brian Bouterse <bbouters at redhat.com>
>>> wrote:
>>>
>>>> Thank you for writing this out. The most significant issue I read in
>>>> this is that 3 of the 9 plugins are having their users take steps that
>>>> aren't adding any value in their workflow. They want to (and have an
>>>> opportunity to) take repository version content and expose it directly.
>>>> They don't need Publications or Publishers. If they need no metadata,
>>>> instead of a "pass-through" publication it would be easier to have a
>>>> Distribution refer to a RepositoryVersion directly and remove the
>>>> "pass-through" feature. They the user could skip this step (
>>>> https://github.com/pulp/pulp_ansible#create-a-publication ) and
>>>> instead post the RepositoryVersion reference as part of the Distributor
>>>> itself (
>>>> https://github.com/pulp/pulp_ansible#create-a-distribution-for-the-publication
>>>> ).
>>>>
>>>> The second issue I read here is that having users CRUD a publisher and
>>>> then call that publisher is not as helpful/useful as CRUDing the resulting
>>>> Publication directly. One practical issue related to this is the saving of
>>>> the "publish" parameters. How do we store those over time? Our modeling
>>>> choices have made this a bit challenging because Publication's aren't
>>>> plugin controlled. A simple remedy is to have plugins provide various
>>>> Publication objects instead of Publishers. These Publications would be
>>>> managed by Master/Detail and when created it would run the task that
>>>> creates the Publication and return the 202. This simplifies Pulp in that
>>>> the options that were used to create the publication can be saved on the
>>>> Publication as provided by various plugins. It also allows users to take
>>>> one less step (they don't need to make a publisher to then make a
>>>> publication). Instead users that need metadata generated can do so with one
>>>> step, making the publication. So my main question is, can anyone remember
>>>> why we didn't use Master/Detail in this area originally?
>>>>
>>>> I believe these are changes we should explore. The top one is a pretty
>>>> simple add on and mostly not a breaking change. The bottom one would be a
>>>> larger change, but one that I believe we could make and should seriously
>>>> consider pre 3.0 GA.
>>>>
>>>>
>>>>
>>>> On Mon, Apr 8, 2019 at 7:42 AM Dennis Kliban <dkliban at redhat.com>
>>>> wrote:
>>>>
>>>>> TLDR:
>>>>>
>>>>> Auto-distribution of publications is performed implicitly instead of
>>>>> explicitly.
>>>>> Plugins that don't generate metadata during publish have to provide a
>>>>> generic publisher.
>>>>> Users have to keep track of publishers to make sure auto-distribution
>>>>> of new publications works.
>>>>>
>>>>> More background:
>>>>>
>>>>> Publishers in Pulp 3 serve three distinct functions:
>>>>>   - store configuration for how a publication should be created
>>>>>   - create publications using the configuration
>>>>>   - update any distributions that are supposed to be auto updated with
>>>>> new publications of a repository (auto distribution).
>>>>>
>>>>> Ansible, Docker, and Maven plugins do not generate any metadata when
>>>>> creating a publication. So their publishers don't need to store any
>>>>> configuration. Users of these plugins only get two benefits from publishers:
>>>>>   - create publications using the configuration
>>>>>   - update any distributions that are supposed to be auto updated with
>>>>> new publications of a repository (auto distribution).
>>>>>
>>>>> The publications created by the publishers of these plugins could be
>>>>> created using a single generic publish task. So the only remaining benefit
>>>>> of a publisher for these plugins is the auto-distiribution that occurs when
>>>>> a Distribution has a repository and a pubisher configured. The only way
>>>>> this feature is documented right now is with the help text on the
>>>>> 'repository' and 'publisher' fields of Distributions.
>>>>>
>>>>> Does anyone else see these as problems with the Publishers?
>>>>> _______________________________________________
>>>>> Pulp-dev mailing list
>>>>> Pulp-dev at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>
>>>> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20190415/caee5634/attachment.htm>