[Pulp-list] Workers disappearing / canceled tasks after upgrading to 2.7.0.1 - CentOS 6.7

[Joel Golden]

Hi Randy,

Here's what I've tried thus far.

1) for i in httpd qpidd pulp_celerybeat pulp_resource_manager pulp_workers;
do service $i restart; done
2) Downgrading to 2.6.4-1.el6
3) Reboot... what?  This isn't Windows.
4) Turned up all logging to debug and can't see what's killing or making
pulp think the resource manager has died.  They never die, they keep their
same time stamp from the restart.
5) Clean build using the same patch set and starting with 2.6.4-1.el6 then
moving to 2.7.0-1.el6.

The only thing that worked the clean build at 2.6.4-1.el6 but only at this
version.  It exhibited the same symptoms as soon as pulp was upgraded.

These are the only packages changed:
Nov 15 21:08:31 Updated: python-pulp-common-2.7.0-1.el6.noarch
Nov 15 21:08:32 Updated: python-pulp-puppet-common-2.7.0-1.el6.noarch
Nov 15 21:08:32 Updated: python-pulp-rpm-common-2.7.0-1.el6.noarch
Nov 15 21:08:32 Updated: python-pulp-bindings-2.7.0-1.el6.noarch
Nov 15 21:08:32 Updated: python-pulp-client-lib-2.7.0-1.el6.noarch
Nov 15 21:08:33 Updated: pulp-admin-client-2.7.0-1.el6.noarch
Nov 15 21:08:33 Installed: python-pulp-repoauth-2.7.0-1.el6.noarch
Nov 15 21:08:33 Installed: python-pulp-oid_validation-2.7.0-1.el6.noarch
Nov 15 21:08:37 Installed: Django14-1.4.21-1.el6.noarch
Nov 15 21:08:46 Updated: pulp-server-2.7.0-1.el6.noarch
Nov 15 21:08:47 Updated: pulp-puppet-plugins-2.7.0-1.el6.noarch
Nov 15 21:08:47 Updated: pulp-rpm-plugins-2.7.0-1.el6.noarch
Nov 15 21:08:47 Updated: pulp-puppet-admin-extensions-2.7.0-1.el6.noarch
Nov 15 21:08:48 Updated: pulp-rpm-admin-extensions-2.7.0-1.el6.noarch
Nov 15 21:08:48 Updated: pulp-selinux-2.7.0-1.el6.noarch
Nov 15 21:08:49 Updated: 1:python-kombu-3.0.24-10.pulp.el6.noarch
Nov 15 21:08:49 Updated: ruby-augeas-0.4.1-3.el6.x86_64
Nov 15 21:08:49 Updated: 1:ruby-shadow-2.2.0-2.el6.x86_64

What is even more perplexing is that I have another CentOS 6.7 box in
production running 2.7.0.1 that isn't having an issue and they were built
following the same procedure.  I could try RabbitMQ but I'd prefer to keep
my lab and production at least running the same components, even though
there must be a version or config difference somewhere.

Thank you,
Joel

On Mon, Nov 16, 2015 at 1:15 PM, Randy Barlow <rbarlow at redhat.com> wrote:

> Hi Joel,
>
> I too have been experiencing symptoms like this on my development
> system. At the moment, we suspect an issue with either qpidd or the
> python-qpid library. Unfortunately, we do not have any updates. In the
> meantime you can try these two workaround:
>
> 0) Try restarting pulp_workers, pulp_resource_manager, and pulp_celerybeat.
>
> 1) Try using RabbitMQ instead of qpidd.
>
> --
> Randy Barlow
> irc:   bowlofeggs
> phone: +1-919-890-8150
>
>
> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20151116/14d9ab0c/attachment.htm>