[Pulp-dev] Proposal: merge the content-app & streamer

[Tatiana Tereshchenko]

+1 to merge
+1 to have clear docs for plugin writers how to create their own content app

On Mon, Dec 3, 2018 at 11:25 PM Dennis Kliban <dkliban at redhat.com> wrote:

> It was pointed out on IRC that plugins that have to supply their own
> content app (such as docker) currently need to supply 2 implementations of
> it in order to support on-demand use cases. One using django and another
> using aiohttp.
>
> We should not burden plugin writers in such a way. We really have to make
> the proposed change.
>
> On Mon, Dec 3, 2018 at 3:24 PM Dana Walker <dawalker at redhat.com> wrote:
>
>> In light of the efficiency gains and lack of significant drawbacks, I'm
>> +1 on this proposal.
>>
>> --Dana
>>
>>
>> Dana Walker
>>
>> Associate Software Engineer
>>
>> Red Hat
>>
>> <https://www.redhat.com>
>> <https://red.ht/sig>
>>
>>
>> On Mon, Dec 3, 2018 at 2:40 PM Dennis Kliban <dkliban at redhat.com> wrote:
>>
>>> I like the idea of combining the two applications for all the reasons
>>> already outlined on this thread. The user experience is going to be
>>> simplified by this change. However, I want to point out that it will also
>>> alter the plugin writer experience. Plugin writers that want to have their
>>> own content app will now need to provide it as a plugin for the content app
>>> (which is not a Django project). We should be able to clearly document this
>>> for plugin writers. pulp_docker plugin will need to adopt this change. For
>>> that reason I'd like us to make a decision on this soon.
>>>
>>> On Fri, Nov 30, 2018 at 4:59 PM Jeff Ortel <jortel at redhat.com> wrote:
>>>
>>>> *BACKGROUND*
>>>>
>>>> The pulp3 content app and the streamer (in-progress) currently have a
>>>> lot of duplicate code and functionality.  At the very least, I think there
>>>> is a opportunity to refactor both and share code.  But, this would leave us
>>>> with two components with significant overlap in functionality.
>>>>
>>>> The functionality exclusive to the content-app:
>>>>   - Optionally delegate file serving to a web server. (Eg:
>>>> mod_xsendfile).
>>>>   - Optional redirect to the streamer.
>>>>
>>>> The functionality exclusive to the streamer:
>>>>   - Using the Remote & RemoteArtifact to download the file and stream
>>>> on demand.
>>>>
>>>> Not much difference which raises the question: "Why do we have both?"
>>>> I think the answer may be that we don't.
>>>>
>>>> *PROPOSAL*
>>>>
>>>> Let's pull the content-app out and merge it with the streamer.  The new
>>>> content (app) would have *streamer* architecture & functionality.
>>>> When a requested artifact has not been downloaded, it would
>>>> download/streamed instead of REDIRECT.  This does mean that deployments and
>>>> development environments would need to run an additional service to serve
>>>> content.  The /pulp/content endpoint would be on a different port than the
>>>> API.  I see this separation as a healthy thing.  There is significant
>>>> efficiency to be gained as well.  Let's start with eliminating the
>>>> REDIRECTs.  Cutting the GET requests in half is a win for both the client,
>>>> the network and the Pulp web stack.  Next is database queries.  Since both
>>>> applications needed to perform many of the same queries, combining the
>>>> applications will roughly cut them in half as well.  Since the streamer is
>>>> based on asyncio and so would the merged app.
>>>>
>>>> There are probably lots of other pros/cons I have not considered but it
>>>> seems relatively straight forward.
>>>>
>>>> I'm thinking the new content app/service would be named: *pulp-content*
>>>> .
>>>>
>>>> Thoughts?
>>>>
>>>> _______________________________________________
>>>> Pulp-dev mailing list
>>>> Pulp-dev at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>
>>> _______________________________________________
>>> Pulp-dev mailing list
>>> Pulp-dev at redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20181204/4177963e/attachment.htm>