[Pulp-dev] Proposal: merge the content-app & streamer
dkliban at redhat.com
Mon Dec 3 22:24:31 UTC 2018
It was pointed out on IRC that plugins that have to supply their own
content app (such as docker) currently need to supply 2 implementations of
it in order to support on-demand use cases. One using django and another
We should not burden plugin writers in such a way. We really have to make
the proposed change.
On Mon, Dec 3, 2018 at 3:24 PM Dana Walker <dawalker at redhat.com> wrote:
> In light of the efficiency gains and lack of significant drawbacks, I'm +1
> on this proposal.
> Dana Walker
> Associate Software Engineer
> Red Hat
> On Mon, Dec 3, 2018 at 2:40 PM Dennis Kliban <dkliban at redhat.com> wrote:
>> I like the idea of combining the two applications for all the reasons
>> already outlined on this thread. The user experience is going to be
>> simplified by this change. However, I want to point out that it will also
>> alter the plugin writer experience. Plugin writers that want to have their
>> own content app will now need to provide it as a plugin for the content app
>> (which is not a Django project). We should be able to clearly document this
>> for plugin writers. pulp_docker plugin will need to adopt this change. For
>> that reason I'd like us to make a decision on this soon.
>> On Fri, Nov 30, 2018 at 4:59 PM Jeff Ortel <jortel at redhat.com> wrote:
>>> The pulp3 content app and the streamer (in-progress) currently have a
>>> lot of duplicate code and functionality. At the very least, I think there
>>> is a opportunity to refactor both and share code. But, this would leave us
>>> with two components with significant overlap in functionality.
>>> The functionality exclusive to the content-app:
>>> - Optionally delegate file serving to a web server. (Eg:
>>> - Optional redirect to the streamer.
>>> The functionality exclusive to the streamer:
>>> - Using the Remote & RemoteArtifact to download the file and stream on
>>> Not much difference which raises the question: "Why do we have both?" I
>>> think the answer may be that we don't.
>>> Let's pull the content-app out and merge it with the streamer. The new
>>> content (app) would have *streamer* architecture & functionality. When
>>> a requested artifact has not been downloaded, it would download/streamed
>>> instead of REDIRECT. This does mean that deployments and development
>>> environments would need to run an additional service to serve content. The
>>> /pulp/content endpoint would be on a different port than the API. I see
>>> this separation as a healthy thing. There is significant efficiency to be
>>> gained as well. Let's start with eliminating the REDIRECTs. Cutting the
>>> GET requests in half is a win for both the client, the network and the Pulp
>>> web stack. Next is database queries. Since both applications needed to
>>> perform many of the same queries, combining the applications will roughly
>>> cut them in half as well. Since the streamer is based on asyncio and so
>>> would the merged app.
>>> There are probably lots of other pros/cons I have not considered but it
>>> seems relatively straight forward.
>>> I'm thinking the new content app/service would be named: *pulp-content*.
>>> Pulp-dev mailing list
>>> Pulp-dev at redhat.com
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Pulp-dev