[Pulp-list] Missing celery workers

Thu Nov 26 05:26:36 UTC 2015

Fix - replace qpid-cpp-server:

Installing:
qpid-cpp-server-linearstore x86_64 0.34-4.el6
replacing qpid-cpp-server-store.x86_64 0.26-9.el6
Updating:
python-qpid noarch 0.32-12.el6
python-qpid-common noarch 0.32-12.el6
python-qpid-qmf x86_64 0.32-1.el6
qpid-cpp-client x86_64 0.34-4.el6
qpid-cpp-server x86_64 0.34-4.el6
qpid-proton-c x86_64 0.10-2.el6
qpid-qmf x86_64 0.32-1.el6
Installing for dependencies:
python-saslwrapper x86_64 0.22-5.el6

Enjoy!  Joel Golden

Due to an existing set of deprecated Qpid packages in RHEL 6 and CentOS 6,
we cannot at present provide updated Qpid packages directly in EPEL 6. We
offer an alternative repository at Fedora Copr.
Qpid at Copr
Copr repo file for RHEL 6 and CentOS 6
https://copr.fedoraproject.org/coprs/irina/qpid/repo/epel-6/irina-qpid-epel-6.repo
Copr Qpid GPG public key
https://qpid.apache.org/copr-qpid-pubkey.gpg

On Thu, Nov 19, 2015 at 5:08 AM, Miller, Jeffrey L <jeff-l-miller at uiowa.edu>
wrote:

> In answer to posed questions:
> - RHEL 6 x86_64
> - qpid-cpp-server
> - Every 60 seconds for me as well. /var/log/messages loos as below.
>
> -Jeffrey
>
>
>
>
> -----Original Message-----
> From: pulp-list-bounces at redhat.com [mailto:pulp-list-bounces at redhat.com]
> On Behalf Of Ashby, Jason (IMS)
> Sent: Wednesday, November 18, 2015 10:38 AM
> To: Brian Bouterse <bbouters at redhat.com>; pulp-list at redhat.com
> Subject: Re: [Pulp-list] Missing celery workers
>
> For me it is consistently every 60 seconds. Here are logs from the last 10
> minutes.  I also set loglevel to DEBUG for each config file in
> /etc/default/pulp* and restarted the services, but I'm not seeing any DEBUG
> stuff in the logs.
>
>
> $ sudo grep missing /var/log/messages
> Nov 18 11:23:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-0 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:23:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-0 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:23:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-2 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:23:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-2 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:23:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-3 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:23:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-3 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:23:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-1 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:23:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-1 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:23:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'resource_manager at pulp01' has gone missing, removing from list of workers
> Nov 18 11:23:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker
> named resource_manager at pulp01 is missing. Canceling the tasks in its
> queue.
> Nov 18 11:24:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-2 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:24:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-2 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:24:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-0 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:24:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-0 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:24:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-3 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:24:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-3 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:24:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-1 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:24:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-1 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:24:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'resource_manager at pulp01' has gone missing, removing from list of workers
> Nov 18 11:24:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker
> named resource_manager at pulp01 is missing. Canceling the tasks in its
> queue.
> Nov 18 11:25:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'resource_manager at pulp01' has gone missing, removing from list of workers
> Nov 18 11:25:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker
> named resource_manager at pulp01 is missing. Canceling the tasks in its
> queue.
> Nov 18 11:25:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-0 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:25:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-0 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:25:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-1 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:25:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-1 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:26:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-0 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:26:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-0 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:26:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-2 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:26:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-2 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:26:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-1 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:26:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-1 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:26:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-3 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:26:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-3 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:26:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'resource_manager at pulp01' has gone missing, removing from list of workers
> Nov 18 11:26:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker
> named resource_manager at pulp01 is missing. Canceling the tasks in its
> queue.
> Nov 18 11:27:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-3 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:27:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-3 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:27:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-0 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:27:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-0 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:27:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-2 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:27:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-2 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:27:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-1 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:27:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-1 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:27:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'resource_manager at pulp01' has gone missing, removing from list of workers
> Nov 18 11:27:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker
> named resource_manager at pulp01 is missing. Canceling the tasks in its
> queue.
> Nov 18 11:28:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-3 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:28:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-3 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:28:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-0 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:28:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-0 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:28:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-2 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:28:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-2 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:28:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-1 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:28:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-1 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:28:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'resource_manager at pulp01' has gone missing, removing from list of workers
> Nov 18 11:28:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker
> named resource_manager at pulp01 is missing. Canceling the tasks in its
> queue.
> Nov 18 11:29:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-0 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:29:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-0 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:29:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-2 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:29:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-2 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:29:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-1 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:29:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-1 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:29:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-3 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:29:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-3 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:29:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'resource_manager at pulp01' has gone missing, removing from list of workers
> Nov 18 11:29:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker
> named resource_manager at pulp01 is missing. Canceling the tasks in its
> queue.
> Nov 18 11:30:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-0 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:30:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-0 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:30:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-2 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:30:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-2 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:30:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-1 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:30:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-1 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:30:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-3 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:30:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-3 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:30:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'resource_manager at pulp01' has gone missing, removing from list of workers
> Nov 18 11:30:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker
> named resource_manager at pulp01 is missing. Canceling the tasks in its
> queue.
> Nov 18 11:31:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-0 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:31:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-0 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:31:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-2 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:31:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-2 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:31:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-1 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:31:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-1 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:31:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-3 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:31:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-3 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:31:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'resource_manager at pulp01' has gone missing, removing from list of workers
> Nov 18 11:31:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker
> named resource_manager at pulp01 is missing. Canceling the tasks in its
> queue.
> Nov 18 11:32:35 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-0 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:32:35 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-0 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:32:35 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-2 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:32:35 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-2 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:32:35 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-1 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:32:35 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-1 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:32:35 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'reserved_resource_worker-3 at pulp01' has gone missing, removing from list
> of workers Nov 18 11:32:35 pulp01 pulp: pulp.server.async.tasks:ERROR: The
> worker named reserved_resource_worker-3 at pulp01 is missing. Canceling the
> tasks in its queue.
> Nov 18 11:32:35 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> 'resource_manager at pulp01' has gone missing, removing from list of workers
> Nov 18 11:32:35 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker
> named resource_manager at pulp01 is missing. Canceling the tasks in its
> queue.
>
> -----Original Message-----
> From: pulp-list-bounces at redhat.com [mailto:pulp-list-bounces at redhat.com]
> On Behalf Of Brian Bouterse
> Sent: Wednesday, November 18, 2015 11:05 AM
> To: pulp-list at redhat.com
> Subject: Re: [Pulp-list] Missing celery workers
>
> Jason and Jeffrey,
>
> Thanks for reporting this. I've written up a bug [0] and I am
> investigating the root cause.
>
> On the bug are you able to leave some answers to these questions?
>
> - Can you confirm that it affects both RabbitMQ and Qpid usage?
> - Can you confirm that the workers "go missing" and then return, and then
> "go missing" in a continuous cycle? I expect it to happen every 90 seconds.
>
> - Jeffrey specifically, what OS are you using?
>
> [0]: https://pulp.plan.io/issues/1380
>
> Thanks,
> Brian
>
> On 11/18/2015 09:33 AM, Miller, Jeffrey L wrote:
> > I am seeing this behavior as well after upgrading from 2.6 to 2.7.
> > However, I am using qpid not rabbitmq.
> >
> >
> >
> > -Jeffrey
> >
> >
> >
> >
> >
> >
> >
> > *From:* pulp-list-bounces at redhat.com
> > [mailto:pulp-list-bounces at redhat.com] *On Behalf Of *Ashby, Jason
> > (IMS)
> > *Sent:* Wednesday, November 18, 2015 8:29 AM
> > *To:* pulp-list at redhat.com
> > *Subject:* [Pulp-list] Missing celery workers
> >
> >
> >
> > Hi all,
> >
> > I'm hitting another issue with the upgrade to Pulp 2.7.0 + changing
> > from qpid to rabbitmq for messaging.  The workers are continuously
> > going missing, every minute or so.  The effect is that the tasks in
> > the task list stay in a Waiting state and are never completed.
> >
> >
> >
> > Rabbitmq looks healthy; I see successful accepted connections per the
> > logs and can see a bunch of connections in the rabbitmq management GUI.
> > I'm kind of stuck as far as troubleshooting goes.  Any tips on what
> > else to investigate?
> >
> >
> >
> > Pulp and rabbitmq servers are both CentOS 6.
> >
> >
> >
> > # /var/log/messages
> >
> > Nov 18 08:53:56 pulp01 pulp: celery.worker.consumer:INFO: missed
> > heartbeat from resource_manager at pulp01
> >
> > Nov 18 09:05:46 pulp01 pulp: pulp.server.async.worker_watcher:INFO:
> > New worker 'reserved_resource_worker-3 at pulp01' discovered
> >
> > Nov 18 09:05:46 pulp01 pulp: pulp.server.async.worker_watcher:INFO:
> > New worker 'reserved_resource_worker-1 at pulp01' discovered
> >
> > Nov 18 09:05:46 pulp01 pulp: pulp.server.async.worker_watcher:INFO:
> > New worker 'reserved_resource_worker-2 at pulp01' discovered
> >
> > Nov 18 09:05:46 pulp01 pulp: pulp.server.async.worker_watcher:INFO:
> > New worker 'reserved_resource_worker-0 at pulp01' discovered
> >
> > Nov 18 09:05:56 pulp01 pulp: pulp.server.async.worker_watcher:INFO:
> > New worker 'resource_manager at pulp01' discovered
> >
> > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> > 'reserved_resource_worker-3 at pulp01' has gone missing, removing from
> > list of work
> >
> > ers
> >
> > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker
> > named reserved_resource_worker-3 at pulp01 is missing. Canceling the
> > tasks in its q
> >
> > ueue.
> >
> > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> > 'reserved_resource_worker-1 at pulp01' has gone missing, removing from
> > list of work
> >
> > ers
> >
> > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker
> > named reserved_resource_worker-1 at pulp01 is missing. Canceling the
> > tasks in its q
> >
> > ueue.
> >
> > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> > 'reserved_resource_worker-2 at pulp01' has gone missing, removing from
> > list of work
> >
> > ers
> >
> > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker
> > named reserved_resource_worker-2 at pulp01 is missing. Canceling the
> > tasks in its q
> >
> > ueue.
> >
> > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> > 'reserved_resource_worker-0 at pulp01' has gone missing, removing from
> > list of work
> >
> > ers
> >
> > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker
> > named reserved_resource_worker-0 at pulp01 is missing. Canceling the
> > tasks in its q
> >
> > ueue.
> >
> > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker
> > 'resource_manager at pulp01' has gone missing, removing from list of
> > workers
> >
> > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker
> > named resource_manager at pulp01 is missing. Canceling the tasks in its
> queue.
> >
> > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.scheduler:ERROR: There
> > are 0 pulp_resource_manager processes running. Pulp will not operate
> > correctly without
> >
> > at least one pulp_resource_mananger process running.
> >
> >
> >
> > ----------------------------------------------------------------------
> > --
> >
> >
> > Information in this e-mail may be confidential. It is intended only
> > for the addressee(s) identified above. If you are not the
> > addressee(s), or an employee or agent of the addressee(s), please note
> > that any dissemination, distribution, or copying of this communication
> > is strictly prohibited. If you have received this e-mail in error,
> > please notify the sender of the error.
> >
> >
> >
> > _______________________________________________
> > Pulp-list mailing list
> > Pulp-list at redhat.com
> > https://www.redhat.com/mailman/listinfo/pulp-list
> >
>
> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
>
> ________________________________
>
> Information in this e-mail may be confidential. It is intended only for
> the addressee(s) identified above. If you are not the addressee(s), or an
> employee or agent of the addressee(s), please note that any dissemination,
> distribution, or copying of this communication is strictly prohibited. If
> you have received this e-mail in error, please notify the sender of the
> error.
>
> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
>
> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20151125/4b7372d5/attachment.htm>