[Pulp-dev] Proposal: merge the content-app & streamer

[Dennis Kliban]

I like the idea of combining the two applications for all the reasons
already outlined on this thread. The user experience is going to be
simplified by this change. However, I want to point out that it will also
alter the plugin writer experience. Plugin writers that want to have their
own content app will now need to provide it as a plugin for the content app
(which is not a Django project). We should be able to clearly document this
for plugin writers. pulp_docker plugin will need to adopt this change. For
that reason I'd like us to make a decision on this soon.

On Fri, Nov 30, 2018 at 4:59 PM Jeff Ortel <jortel at redhat.com> wrote:

> *BACKGROUND*
>
> The pulp3 content app and the streamer (in-progress) currently have a lot
> of duplicate code and functionality.  At the very least, I think there is a
> opportunity to refactor both and share code.  But, this would leave us with
> two components with significant overlap in functionality.
>
> The functionality exclusive to the content-app:
>   - Optionally delegate file serving to a web server. (Eg: mod_xsendfile).
>   - Optional redirect to the streamer.
>
> The functionality exclusive to the streamer:
>   - Using the Remote & RemoteArtifact to download the file and stream on
> demand.
>
> Not much difference which raises the question: "Why do we have both?"  I
> think the answer may be that we don't.
>
> *PROPOSAL*
>
> Let's pull the content-app out and merge it with the streamer.  The new
> content (app) would have *streamer* architecture & functionality.  When a
> requested artifact has not been downloaded, it would download/streamed
> instead of REDIRECT.  This does mean that deployments and development
> environments would need to run an additional service to serve content.  The
> /pulp/content endpoint would be on a different port than the API.  I see
> this separation as a healthy thing.  There is significant efficiency to be
> gained as well.  Let's start with eliminating the REDIRECTs.  Cutting the
> GET requests in half is a win for both the client, the network and the Pulp
> web stack.  Next is database queries.  Since both applications needed to
> perform many of the same queries, combining the applications will roughly
> cut them in half as well.  Since the streamer is based on asyncio and so
> would the merged app.
>
> There are probably lots of other pros/cons I have not considered but it
> seems relatively straight forward.
>
> I'm thinking the new content app/service would be named: *pulp-content*.
>
> Thoughts?
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20181203/f2851a15/attachment.htm>