[Pulp-dev] Proposal: merge the content-app & streamer

[Jeff Ortel]

_BACKGROUND_

The pulp3 content app and the streamer (in-progress) currently have a 
lot of duplicate code and functionality.  At the very least, I think 
there is a opportunity to refactor both and share code.  But, this would 
leave us with two components with significant overlap in functionality.

The functionality exclusive to the content-app:
   - Optionally delegate file serving to a web server. (Eg: mod_xsendfile).
   - Optional redirect to the streamer.

The functionality exclusive to the streamer:
   - Using the Remote & RemoteArtifact to download the file and stream 
on demand.

Not much difference which raises the question: "Why do we have both?"  I 
think the answer may be that we don't.

_PROPOSAL_

Let's pull the content-app out and merge it with the streamer.  The new 
content (app) would have /streamer/ architecture & functionality.  When 
a requested artifact has not been downloaded, it would download/streamed 
instead of REDIRECT.  This does mean that deployments and development 
environments would need to run an additional service to serve content.  
The /pulp/content endpoint would be on a different port than the API.  I 
see this separation as a healthy thing.  There is significant efficiency 
to be gained as well.  Let's start with eliminating the REDIRECTs.  
Cutting the GET requests in half is a win for both the client, the 
network and the Pulp web stack.  Next is database queries.  Since both 
applications needed to perform many of the same queries, combining the 
applications will roughly cut them in half as well.  Since the streamer 
is based on asyncio and so would the merged app.

There are probably lots of other pros/cons I have not considered but it 
seems relatively straight forward.

I'm thinking the new content app/service would be named: /pulp-content/.

Thoughts?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20181130/5c68f299/attachment.htm>