[Pulp-dev] Port Pulp3 to use RQ
Bryan Kearney
bkearney at redhat.com
Tue May 8 12:48:03 UTC 2018
what does this look like for upgrading from Pulp2 to Pulp3?
-- bk
On 05/08/2018 05:34 AM, David Davis wrote:
> +1. Thank you @bmbouter and @dalley for working on this.
>
>
> David
>
> On Mon, May 7, 2018 at 5:37 PM, Daniel Alley <dalley at redhat.com
> <mailto:dalley at redhat.com>> wrote:
>
> I've finished my review and resolved all of the 'blocker' issues
> that were uncovered during testing. Overall, I'm highly confident
> that this is a better path forwards than the continued use of Celery
> / Kombu. There are a couple of outstanding edge cases to be
> resolved eventually, which I plan to file as issues post-merge, but
> nothing serious or intractable.
>
> If there are no objections, I think it would be reasonable to merge
> this code after this week's beta builds are published (after, in
> order to avoid major changes during Summit / PyCon prep time).
>
> Thank you, Brian, for doing the planning and work needed to make
> this happen. It was a lot of effort and is very highly appreciated.
>
> On Mon, Apr 30, 2018 at 8:28 AM, Brian Bouterse <bbouters at redhat.com
> <mailto:bbouters at redhat.com>> wrote:
>
> Through several rebases, now all PRs are showing the RQ PRs on
> Travis as passing with pulp-smash. Several points of feedback
> have been addressed.
>
> If you're interested in commenting on these PRs or trying them
> out, please do. I hope to merge after the other taking system
> maintainers @dalley and @daviddavis have finished their
> testing/review and barring any other calls for delay or blocking
> concerns.
>
> If there are any questions, issues, or concerns, please reach
> out, and we can talk through them.
>
>
>
> On Fri, Apr 20, 2018 at 4:18 PM, Brian Bouterse
> <bbouters at redhat.com <mailto:bbouters at redhat.com>> wrote:
>
> I put together a prototype and posted the PRs. I'm still
> working to get Travis happy, but locally 100% of smash tests
> using these branches. It's worked very reliably for me so
> far. There are no gaps in the pulp feature set on top of RQ.
>
> I hope people test it out and give some feedback. See the
> commit messages for details on what was done. Here are the PRs:
>
> https://github.com/pulp/pulp/pull/3454
> <https://github.com/pulp/pulp/pull/3454>
> https://github.com/pulp/pulp_file/pull/72
> <https://github.com/pulp/pulp_file/pull/72>
> https://github.com/pulp/devel/pull/146
> <https://github.com/pulp/devel/pull/146>
> https://github.com/PulpQE/pulp-smash/pull/960
> <https://github.com/PulpQE/pulp-smash/pull/960>
>
> Feel free to send questions here or to the PR. Any feedback
> is welcome.
>
> -Brian
>
>
>
>
> On Thu, Mar 22, 2018 at 5:28 PM, Milan Kovacik
> <mkovacik at redhat.com <mailto:mkovacik at redhat.com>> wrote:
>
> +1 I like RQ and I like
> http://python-rq.org/docs/testing/
> <http://python-rq.org/docs/testing/> esp.
> there's Fakeredis ;)
>
>
> --
> milan
>
>
> On Thu, Mar 22, 2018 at 6:58 PM, Brian Bouterse
> <bbouters at redhat.com <mailto:bbouters at redhat.com>> wrote:
> > Thanks for all the discussion both on list and on irc.
> After more
> > investigation, it sounds like there are no feature
> gaps, but we will need to
> > incorporate this workaround to cancel a task that is
> already running.
> >
> > The feedback I've heard on the idea is that it's
> valuable and looks
> > feasible, but we won't really know until we prototype
> it a bit. Based on the
> > technical outline in the previous email, I believe it
> can be prototyped in a
> > day or two. I plan to do this soon, once I contribute
> to a few other
> > required-for-beta planning items first. I'll post my
> PR to see what other
> > think of the change, probably next week.
> >
> >
> >
> > On Wed, Mar 21, 2018 at 6:41 PM, Daniel Alley
> <dalley at redhat.com <mailto:dalley at redhat.com>> wrote:
> >>
> >> I meant in the sense that, what is the aftermath when
> it comes back
> >> online, and is it screwed up in ways that cause side
> effects.
> >>
> >> On Wed, Mar 21, 2018 at 5:02 PM, Jeremy Audet
> <jaudet at redhat.com <mailto:jaudet at redhat.com>> wrote:
> >>>
> >>> > RQ does not support revoking tasks. If you send
> the worker a SIGINT,
> >>> > it will finish the task and then stop processing
> new ones. If you send the
> >>> > worker SIGKILL, it will stop immediately, but I
> don't think it gracefully
> >>> > handles this circumstance.
> >>>
> >>> Nothing handles SIGKILL gracefully. Processes can't
> catch that signal.
> >>> `kill -9 $pid` sends SIGKILL.
> >>>
> >>> If one is looking for a way to gracefully,
> immediately kill an RQ
> >>> worker, then SIGTERM may do the trick. Anecdotally,
> many processes
> >>> handle this signal in a hurried fashion.
> Semantically, this is
> >>> appropriate: SIGINT is the "terminal interrupt"
> signal (Ctrl+c sends
> >>> SIGINT), whereas SIGTERM is the "termination signal."
> >>
> >>
> >
> >
> > _______________________________________________
> > Pulp-dev mailing list
> > Pulp-dev at redhat.com <mailto:Pulp-dev at redhat.com>
> > https://www.redhat.com/mailman/listinfo/pulp-dev
> <https://www.redhat.com/mailman/listinfo/pulp-dev>
> >
>
>
>
>
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com <mailto:Pulp-dev at redhat.com>
> https://www.redhat.com/mailman/listinfo/pulp-dev
> <https://www.redhat.com/mailman/listinfo/pulp-dev>
>
>
>
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20180508/ca30d9f4/attachment.sig>
More information about the Pulp-dev
mailing list