[Pulp-dev] Port Pulp3 to use RQ

[Brian Bouterse]

I put together a prototype and posted the PRs. I'm still working to get
Travis happy, but locally 100% of smash tests using these branches. It's
worked very reliably for me so far. There are no gaps in the pulp feature
set on top of RQ.

I hope people test it out and give some feedback. See the commit messages
for details on what was done. Here are the PRs:

https://github.com/pulp/pulp/pull/3454
https://github.com/pulp/pulp_file/pull/72
https://github.com/pulp/devel/pull/146
https://github.com/PulpQE/pulp-smash/pull/960

Feel free to send questions here or to the PR. Any feedback is welcome.

-Brian




On Thu, Mar 22, 2018 at 5:28 PM, Milan Kovacik <mkovacik at redhat.com> wrote:

> +1 I like RQ and I like http://python-rq.org/docs/testing/ esp.
> there's Fakeredis ;)
>
>
> --
> milan
>
>
> On Thu, Mar 22, 2018 at 6:58 PM, Brian Bouterse <bbouters at redhat.com>
> wrote:
> > Thanks for all the discussion both on list and on irc. After more
> > investigation, it sounds like there are no feature gaps, but we will
> need to
> > incorporate this workaround to cancel a task that is already running.
> >
> > The feedback I've heard on the idea is that it's valuable and looks
> > feasible, but we won't really know until we prototype it a bit. Based on
> the
> > technical outline in the previous email, I believe it can be prototyped
> in a
> > day or two. I plan to do this soon, once I contribute to a few other
> > required-for-beta planning items first. I'll post my PR to see what other
> > think of the change, probably next week.
> >
> >
> >
> > On Wed, Mar 21, 2018 at 6:41 PM, Daniel Alley <dalley at redhat.com> wrote:
> >>
> >> I meant in the sense that, what is the aftermath when it comes back
> >> online, and is it screwed up in ways that cause side effects.
> >>
> >> On Wed, Mar 21, 2018 at 5:02 PM, Jeremy Audet <jaudet at redhat.com>
> wrote:
> >>>
> >>> > RQ does not support revoking tasks.  If you send the worker a SIGINT,
> >>> > it will finish the task and then stop processing new ones.  If you
> send the
> >>> > worker SIGKILL, it will stop immediately, but I don't think it
> gracefully
> >>> > handles this circumstance.
> >>>
> >>> Nothing handles SIGKILL gracefully. Processes can't catch that signal.
> >>> `kill -9 $pid` sends SIGKILL.
> >>>
> >>> If one is looking for a way to gracefully, immediately kill an RQ
> >>> worker, then SIGTERM may do the trick. Anecdotally, many processes
> >>> handle this signal in a hurried fashion. Semantically, this is
> >>> appropriate: SIGINT is the "terminal interrupt" signal (Ctrl+c sends
> >>> SIGINT), whereas SIGTERM is the "termination signal."
> >>
> >>
> >
> >
> > _______________________________________________
> > Pulp-dev mailing list
> > Pulp-dev at redhat.com
> > https://www.redhat.com/mailman/listinfo/pulp-dev
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20180420/d43b1950/attachment.htm>