[Pulp-dev] Basic Lazy Streaming for Pulp3 Ready for Rough Testing

[Jeff Ortel]

The initial planning for lazy omitted content protection for 3.0. Since 
then, we have pulled content protection back into 3.0 re: 
content-guards.  In pulp2, the content app redirected using a signed-url 
so that clients could not circumvent content protection. Currently in 
3.0, there is nothing to keep clients from circumventing content 
protection by going directly to the streamer. Isn't this a gap?


On 11/20/18 3:51 PM, Brian Bouterse wrote:
> I've been developing the streamer functionality, and it's correctly 
> working (in my testing) as driven from the Remote.policy attribute. It 
> correctly works with 'immediate', 'on_demand', and 'cache_only'. Read 
> more about the expected behaviors in the epic [0].
>
> # Try it out!
> Here is the core commit needed: https://github.com/pulp/pulp/pull/3738
> Here is the streamer you should pip install from master: 
> https://github.com/bmbouter/pulp_streamer
> Here is what it looks like to port a plugin using DeclarativeVersion, 
> e.g. pulp_file to support lazy: https://github.com/pulp/pulp_file/pull/132
>
> You'll need to configure Pulp's webserver for streaming. I did this by 
> exporting an environment var to dynaconf in the same bash environment 
> as my django run server. Specifically I configured Pulp to redirect to 
> port localhost:8080/streamer/ with this command:
>
> export PULP_CONTENT='@json {"HOST": null, "WEB_SERVER": "django", 
> "REDIRECT": {"ENABLED": true, "PORT": 8080, "HOST": "localhost", 
> "PATH_PREFIX": "/streamer/"}}'
>
> Then I run the streamer (after pip installed) with gunicorn which you 
> also need to pip install. Run it with:
>
> gunicorn pulpcore.streamer:server --bind localhost:8080 --worker-class 
> aiohttp.GunicornWebWorker -w 2
>
> Then sync a pulp_file repo with policy='on_demand' or 
> policy='cache_only' and see how Pulp behaves.
>
> Feedback, ideas, concerns are welcome in any form. Note this is still 
> rough, and the following are known things to be done:
>
> * fix tests to get Travis passing
> * docs for the streamer and for pulpcore
> * an installer role to install the streamer
> * integration with squid to cache lots of data at the streamer
> * transfer the pulp_streamer to the Pulp org on github
> * publish an initial release to PyPI for users to use it
> * write a blog post about porting to it and using it
> * make a demo
>
> [0]: https://pulp.plan.io/issues/3693
>
> Thanks!
> Brian
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20181126/203f3ed0/attachment.htm>