[Pulp-list] Celery and the death of pulp tasking and gofer

[Jay Dobies]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 01/18/2011 04:56 PM, Jason L Connor wrote:
> Hi All,
> 
> I've been watching this project grow for the last 6 months or so, and I
> think it's time to seriously consider it.
> 
> Celery is a distributed task queue framework for python. It can use an
> number of back ends, including mongodb and rabbitmq. It allows tasks to
> be run synchronously and asynchronously, locally and remotely. It has
> task persistence already implemented, as well as scheduling, including
> retries.
> 
> It now runs on python 2.4+ (which it didn't used to), making it meet our
> need to be rhel 5 compatible.
> 
> It's home page is here:
> http://celeryproject.org/
> 
> I propose we take a very serious look at celery to replace both the pulp
> tasking libraries and gofer project for our asynchronous and remote
> execution needs.
> 
> It's feature set is well beyond what we've achieved, and I believe in a
> single sprint we could replace the existing libraries and lend our
> creative talents to a project already implementing the hard bits instead
> of rolling our own here.

+1 for pretty much all the reasons Jason listed. We have so much to
accomplish that we don't need to spend any more time reinventing the
wheel with tasking and gofer (both of which still have a number of
outstanding tasks and neither of which are so ingrained in our stuff
that it'd be catastrophic to replace now). Not to mention the stability
gains of using a 3rd party project that's got a lot more time under its
belt developing/testing this sort of stuff than we do.

I definitely think we should do some investigation into this before we
proceed too much further down the tasking and gofer paths.

> 
> 
> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list


- -- 
Jay Dobies
RHCE# 805008743336126
Freenode: jdob
http://pulpproject.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.14 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJNNhGgAAoJEOMmcTqOSQHCal0H+wdeMGx0iNEo5vgsCVpH0C+w
8wl4gsfKWCg14+VNHAKQfd0+AWv9x3hruJgE4gCYEQdp7tEjruvA2o1RCx8OjDfC
pmsrmET537P9ELE60pnX/+MJzcWqsiPsYo5spcRYTORDbc8LbIkUoUDOJxPazTDW
ZpL26uHn1gVMekPWfNvxI2UBQ53vsYLPGKzTPsrh9Nn182uD5nMC5RJTKtLnKrnF
4M+BAGcyZhNn3a2e1qnOl4aaPB1Oe1yBAjNzqxfe0vTyoW68+EvKkwH0JEqxDufh
RYtZ7O3lJWJPhbMoagqxDbqysKakozzQGw8HQQzcQHAPD5PC5wxYGpteVJwIUW8=
=stIR
-----END PGP SIGNATURE-----