[Pulp-dev] Port Pulp3 to use RQ
Jeff Ortel
jortel at redhat.com
Tue May 8 15:12:26 UTC 2018
+1. Thank you @bmbouter and @dalley for working on this.
On 05/08/2018 07:34 AM, David Davis wrote:
> +1. Thank you @bmbouter and @dalley for working on this.
>
>
> David
>
> On Mon, May 7, 2018 at 5:37 PM, Daniel Alley <dalley at redhat.com
> <mailto:dalley at redhat.com>> wrote:
>
> I've finished my review and resolved all of the 'blocker' issues
> that were uncovered during testing. Overall, I'm highly confident
> that this is a better path forwards than the continued use of
> Celery / Kombu. There are a couple of outstanding edge cases to
> be resolved eventually, which I plan to file as issues post-merge,
> but nothing serious or intractable.
>
> If there are no objections, I think it would be reasonable to
> merge this code after this week's beta builds are published
> (after, in order to avoid major changes during Summit / PyCon prep
> time).
>
> Thank you, Brian, for doing the planning and work needed to make
> this happen. It was a lot of effort and is very highly appreciated.
>
> On Mon, Apr 30, 2018 at 8:28 AM, Brian Bouterse
> <bbouters at redhat.com <mailto:bbouters at redhat.com>> wrote:
>
> Through several rebases, now all PRs are showing the RQ PRs on
> Travis as passing with pulp-smash. Several points of feedback
> have been addressed.
>
> If you're interested in commenting on these PRs or trying them
> out, please do. I hope to merge after the other taking system
> maintainers @dalley and @daviddavis have finished their
> testing/review and barring any other calls for delay or
> blocking concerns.
>
> If there are any questions, issues, or concerns, please reach
> out, and we can talk through them.
>
>
>
> On Fri, Apr 20, 2018 at 4:18 PM, Brian Bouterse
> <bbouters at redhat.com <mailto:bbouters at redhat.com>> wrote:
>
> I put together a prototype and posted the PRs. I'm still
> working to get Travis happy, but locally 100% of smash
> tests using these branches. It's worked very reliably for
> me so far. There are no gaps in the pulp feature set on
> top of RQ.
>
> I hope people test it out and give some feedback. See the
> commit messages for details on what was done. Here are the
> PRs:
>
> https://github.com/pulp/pulp/pull/3454
> <https://github.com/pulp/pulp/pull/3454>
> https://github.com/pulp/pulp_file/pull/72
> <https://github.com/pulp/pulp_file/pull/72>
> https://github.com/pulp/devel/pull/146
> <https://github.com/pulp/devel/pull/146>
> https://github.com/PulpQE/pulp-smash/pull/960
> <https://github.com/PulpQE/pulp-smash/pull/960>
>
> Feel free to send questions here or to the PR. Any
> feedback is welcome.
>
> -Brian
>
>
>
>
> On Thu, Mar 22, 2018 at 5:28 PM, Milan Kovacik
> <mkovacik at redhat.com <mailto:mkovacik at redhat.com>> wrote:
>
> +1 I like RQ and I like
> http://python-rq.org/docs/testing/
> <http://python-rq.org/docs/testing/> esp.
> there's Fakeredis ;)
>
>
> --
> milan
>
>
> On Thu, Mar 22, 2018 at 6:58 PM, Brian Bouterse
> <bbouters at redhat.com <mailto:bbouters at redhat.com>> wrote:
> > Thanks for all the discussion both on list and on
> irc. After more
> > investigation, it sounds like there are no feature
> gaps, but we will need to
> > incorporate this workaround to cancel a task that is
> already running.
> >
> > The feedback I've heard on the idea is that it's
> valuable and looks
> > feasible, but we won't really know until we
> prototype it a bit. Based on the
> > technical outline in the previous email, I believe
> it can be prototyped in a
> > day or two. I plan to do this soon, once I
> contribute to a few other
> > required-for-beta planning items first. I'll post my
> PR to see what other
> > think of the change, probably next week.
> >
> >
> >
> > On Wed, Mar 21, 2018 at 6:41 PM, Daniel Alley
> <dalley at redhat.com <mailto:dalley at redhat.com>> wrote:
> >>
> >> I meant in the sense that, what is the aftermath
> when it comes back
> >> online, and is it screwed up in ways that cause
> side effects.
> >>
> >> On Wed, Mar 21, 2018 at 5:02 PM, Jeremy Audet
> <jaudet at redhat.com <mailto:jaudet at redhat.com>> wrote:
> >>>
> >>> > RQ does not support revoking tasks. If you send
> the worker a SIGINT,
> >>> > it will finish the task and then stop processing
> new ones. If you send the
> >>> > worker SIGKILL, it will stop immediately, but I
> don't think it gracefully
> >>> > handles this circumstance.
> >>>
> >>> Nothing handles SIGKILL gracefully. Processes
> can't catch that signal.
> >>> `kill -9 $pid` sends SIGKILL.
> >>>
> >>> If one is looking for a way to gracefully,
> immediately kill an RQ
> >>> worker, then SIGTERM may do the trick.
> Anecdotally, many processes
> >>> handle this signal in a hurried fashion.
> Semantically, this is
> >>> appropriate: SIGINT is the "terminal interrupt"
> signal (Ctrl+c sends
> >>> SIGINT), whereas SIGTERM is the "termination signal."
> >>
> >>
> >
> >
> > _______________________________________________
> > Pulp-dev mailing list
> > Pulp-dev at redhat.com <mailto:Pulp-dev at redhat.com>
> > https://www.redhat.com/mailman/listinfo/pulp-dev
> <https://www.redhat.com/mailman/listinfo/pulp-dev>
> >
>
>
>
>
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com <mailto:Pulp-dev at redhat.com>
> https://www.redhat.com/mailman/listinfo/pulp-dev
> <https://www.redhat.com/mailman/listinfo/pulp-dev>
>
>
>
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20180508/73ac74da/attachment.htm>
More information about the Pulp-dev
mailing list