<div dir="ltr"> <div class="gmail_extra"> <div class="gmail_quote">On Tue, May 8, 2018 at 11:53 AM, Bryan Kearney <<a href="mailto:bkearney@redhat.com" target="_blank">bkearney@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">ok.. so it is quiesce the old system and then normal updates? </blockquote><div>Yes, exactly. :) </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> -- bk On 05/08/2018 07:27 AM, Brian Bouterse wrote: > Here is a look at what the Pulp2 -> Pulp3 necessary things would be > w.r.t this change. These could also be automated. > > 1. Empty the Pulp system of all of it's tasks and stop all Pulp services. > 2. Uninstall RabbitMQ or Qpid if its only purpose was to serve the Pulp > tasking system. (Satellite uses Qpid in other ways so it would likely > keep it in the architecture for other purposes). > 3. Uninstall Celery/kombu/billiard/py-amqp (the whole celery stack > effectively) > 4. upgrade the bits to Pulp3. This will also bring RQ with it automatically. > 5. Install Redis as a new service in your infra > 6. Replace the systemd files for the pulp_workers and > pulp_resource_manager. This causes systemd to start RQ instead of Celery. > 7. [optional] Configure Redis auth/ssl and configure Pulp's settings > file to match if that is part of your environment. > > Questions/ideas/concerns are welcome. > > > > On Tue, May 8, 2018 at 8:48 AM, Bryan Kearney <<a href="mailto:bkearney@redhat.com">bkearney@redhat.com</a> > <mailto:<a href="mailto:bkearney@redhat.com">bkearney@redhat.com</a>>> wrote: > > what does this look like for upgrading from Pulp2 to Pulp3? > > -- bk > > On 05/08/2018 05:34 AM, David Davis wrote: > > +1. Thank you @bmbouter and @dalley for working on this. > > > > > > David > > > > On Mon, May 7, 2018 at 5:37 PM, Daniel Alley <<a href="mailto:dalley@redhat.com">dalley@redhat.com</a> <mailto:<a href="mailto:dalley@redhat.com">dalley@redhat.com</a>> > > <mailto:<a href="mailto:dalley@redhat.com">dalley@redhat.com</a> <mailto:<a href="mailto:dalley@redhat.com">dalley@redhat.com</a>>>> wrote: > > > > I've finished my review and resolved all of the 'blocker' issues > > that were uncovered during testing. Overall, I'm highly confident > > that this is a better path forwards than the continued use of Celery > > / Kombu. There are a couple of outstanding edge cases to be > > resolved eventually, which I plan to file as issues post-merge, but > > nothing serious or intractable. > > > > If there are no objections, I think it would be reasonable to merge > > this code after this week's beta builds are published (after, in > > order to avoid major changes during Summit / PyCon prep time). > > > > Thank you, Brian, for doing the planning and work needed to make > > this happen. It was a lot of effort and is very highly appreciated. > > > > On Mon, Apr 30, 2018 at 8:28 AM, Brian Bouterse <<a href="mailto:bbouters@redhat.com">bbouters@redhat.com</a> <mailto:<a href="mailto:bbouters@redhat.com">bbouters@redhat.com</a>> > > <mailto:<a href="mailto:bbouters@redhat.com">bbouters@redhat.com</a> <mailto:<a href="mailto:bbouters@redhat.com">bbouters@redhat.com</a>>>> wrote: > > > > Through several rebases, now all PRs are showing the RQ PRs on > > Travis as passing with pulp-smash. Several points of feedback > > have been addressed. > > > > If you're interested in commenting on these PRs or trying them > > out, please do. I hope to merge after the other taking system > > maintainers @dalley and @daviddavis have finished their > > testing/review and barring any other calls for delay or blocking > > concerns. > > > > If there are any questions, issues, or concerns, please reach > > out, and we can talk through them. > > > > > > > > On Fri, Apr 20, 2018 at 4:18 PM, Brian Bouterse > > <<a href="mailto:bbouters@redhat.com">bbouters@redhat.com</a> <mailto:<a href="mailto:bbouters@redhat.com">bbouters@redhat.com</a>> <div><div class="h5">> <mailto:<a href="mailto:bbouters@redhat.com">bbouters@redhat.com</a> <mailto:<a href="mailto:bbouters@redhat.com">bbouters@redhat.com</a>>>> wrote: > > > > I put together a prototype and posted the PRs. I'm still > > working to get Travis happy, but locally 100% of smash tests > > using these branches. It's worked very reliably for me so > > far. There are no gaps in the pulp feature set on top of RQ. > > > > I hope people test it out and give some feedback. See the > > commit messages for details on what was done. Here are the PRs: > > > > <a href="https://github.com/pulp/pulp/pull/3454" rel="noreferrer" target="_blank">https://github.com/pulp/pulp/pull/3454</a> > <<a href="https://github.com/pulp/pulp/pull/3454" rel="noreferrer" target="_blank">https://github.com/pulp/pulp/pull/3454</a>> > > <<a href="https://github.com/pulp/pulp/pull/3454" rel="noreferrer" target="_blank">https://github.com/pulp/pulp/pull/3454</a> > <<a href="https://github.com/pulp/pulp/pull/3454" rel="noreferrer" target="_blank">https://github.com/pulp/pulp/pull/3454</a>>> > > <a href="https://github.com/pulp/pulp_file/pull/72" rel="noreferrer" target="_blank">https://github.com/pulp/pulp_file/pull/72</a> > <<a href="https://github.com/pulp/pulp_file/pull/72" rel="noreferrer" target="_blank">https://github.com/pulp/pulp_file/pull/72</a>> > > <<a href="https://github.com/pulp/pulp_file/pull/72" rel="noreferrer" target="_blank">https://github.com/pulp/pulp_file/pull/72</a> > <<a href="https://github.com/pulp/pulp_file/pull/72" rel="noreferrer" target="_blank">https://github.com/pulp/pulp_file/pull/72</a>>> > > <a href="https://github.com/pulp/devel/pull/146" rel="noreferrer" target="_blank">https://github.com/pulp/devel/pull/146</a> > <<a href="https://github.com/pulp/devel/pull/146" rel="noreferrer" target="_blank">https://github.com/pulp/devel/pull/146</a>> > > <<a href="https://github.com/pulp/devel/pull/146" rel="noreferrer" target="_blank">https://github.com/pulp/devel/pull/146</a> > <<a href="https://github.com/pulp/devel/pull/146" rel="noreferrer" target="_blank">https://github.com/pulp/devel/pull/146</a>>> > > <a href="https://github.com/PulpQE/pulp-smash/pull/960" rel="noreferrer" target="_blank">https://github.com/PulpQE/pulp-smash/pull/960</a> > <<a href="https://github.com/PulpQE/pulp-smash/pull/960" rel="noreferrer" target="_blank">https://github.com/PulpQE/pulp-smash/pull/960</a>> > > <<a href="https://github.com/PulpQE/pulp-smash/pull/960" rel="noreferrer" target="_blank">https://github.com/PulpQE/pulp-smash/pull/960</a> > <<a href="https://github.com/PulpQE/pulp-smash/pull/960" rel="noreferrer" target="_blank">https://github.com/PulpQE/pulp-smash/pull/960</a>>> > > > > Feel free to send questions here or to the PR. Any feedback > > is welcome. > > > > -Brian > > > > > > > > > > On Thu, Mar 22, 2018 at 5:28 PM, Milan Kovacik > > <<a href="mailto:mkovacik@redhat.com">mkovacik@redhat.com</a> <mailto:<a href="mailto:mkovacik@redhat.com">mkovacik@redhat.com</a>> </div></div>> <mailto:<a href="mailto:mkovacik@redhat.com">mkovacik@redhat.com</a> <mailto:<a href="mailto:mkovacik@redhat.com">mkovacik@redhat.com</a>>>> wrote: > > > > +1 I like RQ and I like > > <a href="http://python-rq.org/docs/testing/" rel="noreferrer" target="_blank">http://python-rq.org/docs/testing/</a> <<a href="http://python-rq.org/docs/testing/" rel="noreferrer" target="_blank">http://python-rq.org/docs/testing/</a>> > > <<a href="http://python-rq.org/docs/testing/" rel="noreferrer" target="_blank">http://python-rq.org/docs/testing/</a> > <<a href="http://python-rq.org/docs/testing/" rel="noreferrer" target="_blank">http://python-rq.org/docs/testing/</a>>> esp. > > there's Fakeredis ;) > > > > > > -- > > milan > > > > > > On Thu, Mar 22, 2018 at 6:58 PM, Brian Bouterse > > <<a href="mailto:bbouters@redhat.com">bbouters@redhat.com</a> <mailto:<a href="mailto:bbouters@redhat.com">bbouters@redhat.com</a>> > <mailto:<a href="mailto:bbouters@redhat.com">bbouters@redhat.com</a> <mailto:<a href="mailto:bbouters@redhat.com">bbouters@redhat.com</a>>>> wrote: > > > Thanks for all the discussion both on list and on irc. > > After more > > > investigation, it sounds like there are no feature > > gaps, but we will need to > > > incorporate this workaround to cancel a task that is > > already running. > > > > > > The feedback I've heard on the idea is that it's > > valuable and looks > > > feasible, but we won't really know until we prototype > > it a bit. Based on the > > > technical outline in the previous email, I believe it > > can be prototyped in a > > > day or two. I plan to do this soon, once I contribute > > to a few other > > > required-for-beta planning items first. I'll post my > > PR to see what other > > > think of the change, probably next week. > > > > > > > > > > > > On Wed, Mar 21, 2018 at 6:41 PM, Daniel Alley > > <<a href="mailto:dalley@redhat.com">dalley@redhat.com</a> <mailto:<a href="mailto:dalley@redhat.com">dalley@redhat.com</a>> > <mailto:<a href="mailto:dalley@redhat.com">dalley@redhat.com</a> <mailto:<a href="mailto:dalley@redhat.com">dalley@redhat.com</a>>>> wrote: > > >> > > >> I meant in the sense that, what is the aftermath when > > it comes back > > >> online, and is it screwed up in ways that cause side > > effects. > > >> > > >> On Wed, Mar 21, 2018 at 5:02 PM, Jeremy Audet > > <<a href="mailto:jaudet@redhat.com">jaudet@redhat.com</a> <mailto:<a href="mailto:jaudet@redhat.com">jaudet@redhat.com</a>> > <mailto:<a href="mailto:jaudet@redhat.com">jaudet@redhat.com</a> <mailto:<a href="mailto:jaudet@redhat.com">jaudet@redhat.com</a>>>> wrote: > > >>> > > >>> > RQ does not support revoking tasks. If you send > > the worker a SIGINT, > > >>> > it will finish the task and then stop processing > > new ones. If you send the > > >>> > worker SIGKILL, it will stop immediately, but I > > don't think it gracefully > > >>> > handles this circumstance. > > >>> > > >>> Nothing handles SIGKILL gracefully. Processes can't > > catch that signal. > > >>> `kill -9 $pid` sends SIGKILL. > > >>> > > >>> If one is looking for a way to gracefully, > > immediately kill an RQ > > >>> worker, then SIGTERM may do the trick. Anecdotally, > > many processes > > >>> handle this signal in a hurried fashion. > > Semantically, this is > > >>> appropriate: SIGINT is the "terminal interrupt" > > signal (Ctrl+c sends > > >>> SIGINT), whereas SIGTERM is the "termination signal." > > >> > > >> > > > > > > > > > _______________________________________________ > > > Pulp-dev mailing list > > > <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>> > <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>>> > > > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>> > > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>>> > > > > > > > > > > > > > > > _______________________________________________ > > Pulp-dev mailing list > > <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>> > <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>>> <div class="HOEnZb"><div class="h5">> > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>> > > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>>> > > > > > > > > > > _______________________________________________ > > Pulp-dev mailing list > > <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>> > > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>> > > > > > > _______________________________________________ > Pulp-dev mailing list > <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>> > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>> > > </div></div></blockquote></div> </div></div>