[Pulp-dev] proposal: global importer settings

[Michael Hrivnak]

Bump. Any more questions on this? I'd like to get something agreed on and
settled.

Thanks!



On Fri, Nov 3, 2017 at 1:28 PM, Michael Hrivnak <mhrivnak at redhat.com> wrote:

>
>
> On Fri, Nov 3, 2017 at 11:58 AM, Jeff Ortel <jortel at redhat.com> wrote:
>
>>
>>
>> On 11/02/2017 10:25 AM, Michael Hrivnak wrote:
>> > I've been working on a planning task for how Pulp 3 will handle global
>> importer settings. As part of that,
>> > I've collected feedback from a number of stakeholders. You can view the
>> planning task here:
>> >
>> > https://pulp.plan.io/issues/2373
>> >
>> > The aspect that everyone seems to agree on is that proxies should be
>> configured once in one place, and that
>> > other download-related settings are a good fit for the same mechanism.
>> I wrote up a story for that here, and I
>> > would appreciate feedback and grooming:
>> >
>> > https://pulp.plan.io/issues/3108
>> >
>> > There is much less clarity around other settings. This is because most
>> other settings that we would consider
>> > for global scope would need the ability to be overridden at the
>> individual importer level. That multi-layered
>> > approach where an individual importer's settings take precedence over
>> the global settings is how Pulp 2 works,
>> > but it is not generally liked. Katello for example only uses that
>> feature in Pulp 2 to define download-related
>> > settings; proxy, concurrency, and bandwidth limits. Many stakeholders
>> expressed concern about retaining a
>> > multi-layered approach to configuring importers. Doing so also would
>> add substantial complexity to how Pulp 3
>> > implements settings, and it adds complexity to the user experience.
>>
>> Can you elaborate on what pulp2 users don't like about the
>> "multi-layered" approach in pulp2?
>>
>> IIRC, pulp2 only supports the "override config" thing (which I agree, few
>> users like) and not something
>> persistent.  I suspect that users don't like the pulp2 implementation but
>> may find the more sophisticated
>> profile concept appealing even with a precedence model.  Not supporting a
>> model where the importer attributes
>> take precedence over the profile will likely limit the properties in the
>> profile to just the proxy.
>>
>> This could be a more powerful feature.  I'm imagining cases where users
>> would find it useful for the profile
>> to include properties beyond proxy URL (or other download properties).
>> Perhaps even the download or sync
>> policies.  Who knows.
>>
>> I do recognize that we really only have a concrete use case for the proxy
>> but seems short sighted to design
>> what seems to be intended as a more general concept that would be so
>> limited.
>>
>> I don't perceive a precedence model as being particularly complex.
>
>
> Pulp 2 has a three-layer approach. You can define importer settings
> globally in a json file on disk, on an importer record in the DB, and in an
> override config passed at the time of requesting an individual sync. That
> contributes to it being difficult to know what settings actually got used
> for any particular sync. And since the same setting can be set in multiple
> places, users are sometimes surprised that a sync doesn't behave the way
> they expected.
>
> I think there is a place for having global importer settings that
> influence how the imports happen. The download policy (immediate,
> on-demand, etc) is a good example of something a user may want to set
> globally. I think we could figure out a reasonable 2-layer approach to
> doing that. But since importers are a master/detail model, that introduces
> some complexity. And I do not see any particular reason we would need to
> deliver this feature in 3.0.
>
> Download settings are different, because I think the best way to model
> them is in one place without any layering. And from a use-case standpoint,
> users will logically group download settings differently than importer
> settings. For example:
>
> I want to use a proxy for external traffic, a different one for traffic to
> a data center, and none for internal traffic. It's all about physical
> network characteristics, and not at all about content and repos.
>
> I want to set a download policy, or other importer settings, based on
> other factors. Certain vendors, or certain products/repos within a specific
> vendor, should get the same download policy based on things like how likely
> I think it is that I'll need all the content in the repo, how reliable I
> think that vendor will be at providing it on-demand, whether I expect to
> test it up-front, etc.
>
> So based on that, I think these are two different kinds of data that we
> should handle separately. We can make a DownloadProfile with a well-defined
> purpose, and that has the single source of data for any attributes it
> stores. Later we could figure out how to make importer settings
> configurable globally by adding a second layer, enabling master/detail with
> that layer, etc.
>
> --
>
> Michael Hrivnak
>
> Principal Software Engineer, RHCE
>
> Red Hat
>



-- 

Michael Hrivnak

Principal Software Engineer, RHCE

Red Hat
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20171114/b1a6591a/attachment.htm>