<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <meta name="Generator" content="Microsoft Word 15 (filtered medium)"> <style></style> </head> <body lang="EN-US" link="blue" vlink="purple"> <div class="WordSection1"> Hmm, I don’t think this would fix the missing worker issue for people like myself using rabbitmq instead of qpid. Though it may fix the other issue of broken EL6 qpid dependencies.<o:p></o:p> <o:p> </o:p> From: pulp-list-bounces@redhat.com [mailto:pulp-list-bounces@redhat.com] On Behalf Of Joel Golden Sent: Thursday, November 26, 2015 12:27 AM To: pulp-list@redhat.com Subject: Re: [Pulp-list] Missing celery workers<o:p></o:p> <o:p> </o:p> <div> <div> Fix - replace qpid-cpp-server:<o:p></o:p> </div> <div> <o:p> </o:p> </div> <div> Installing: <o:p></o:p> </div> <div> qpid-cpp-server-linearstore x86_64 0.34-4.el6<o:p></o:p> </div> <div> replacing qpid-cpp-server-store.x86_64 0.26-9.el6<o:p></o:p> </div> <div> Updating: <o:p></o:p> </div> <div> python-qpid noarch 0.32-12.el6<o:p></o:p> </div> <div> python-qpid-common noarch 0.32-12.el6<o:p></o:p> </div> <div> python-qpid-qmf x86_64 0.32-1.el6<o:p></o:p> </div> <div> qpid-cpp-client x86_64 0.34-4.el6<o:p></o:p> </div> <div> qpid-cpp-server x86_64 0.34-4.el6<o:p></o:p> </div> <div> qpid-proton-c x86_64 0.10-2.el6<o:p></o:p> </div> <div> qpid-qmf x86_64 0.32-1.el6<o:p></o:p> </div> <div> Installing for dependencies:<o:p></o:p> </div> <div> python-saslwrapper x86_64 0.22-5.el6<o:p></o:p> </div> <div> <o:p> </o:p> </div> <div> Enjoy! Joel Golden<o:p></o:p> </div> <div> <o:p> </o:p> </div> <div> Due to an existing set of deprecated Qpid packages in RHEL 6 and CentOS 6, we cannot at present provide updated Qpid packages directly in EPEL 6. We offer an alternative repository at Fedora Copr.<o:p></o:p> </div> <div> Qpid at Copr <o:p></o:p> </div> <div> Copr repo file for RHEL 6 and CentOS 6<o:p></o:p> </div> <div> <a href="https://copr.fedoraproject.org/coprs/irina/qpid/repo/epel-6/irina-qpid-epel-6.repo">https://copr.fedoraproject.org/coprs/irina/qpid/repo/epel-6/irina-qpid-epel-6.repo</a><o:p></o:p> </div> <div> Copr Qpid GPG public key<o:p></o:p> </div> <div> <a href="https://qpid.apache.org/copr-qpid-pubkey.gpg">https://qpid.apache.org/copr-qpid-pubkey.gpg</a><o:p></o:p> </div> </div> <div> <o:p> </o:p> <div> On Thu, Nov 19, 2015 at 5:08 AM, Miller, Jeffrey L <<a href="mailto:jeff-l-miller@uiowa.edu" target="_blank">jeff-l-miller@uiowa.edu</a>> wrote:<o:p></o:p> <blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in"> In answer to posed questions: - RHEL 6 x86_64 - qpid-cpp-server - Every 60 seconds for me as well. /var/log/messages loos as below. -Jeffrey<o:p></o:p> <div> <div> -----Original Message----- From: <a href="mailto:pulp-list-bounces@redhat.com">pulp-list-bounces@redhat.com</a> [mailto:<a href="mailto:pulp-list-bounces@redhat.com">pulp-list-bounces@redhat.com</a>] On Behalf Of Ashby, Jason (IMS) Sent: Wednesday, November 18, 2015 10:38 AM To: Brian Bouterse <<a href="mailto:bbouters@redhat.com">bbouters@redhat.com</a>>; <a href="mailto:pulp-list@redhat.com">pulp-list@redhat.com</a> Subject: Re: [Pulp-list] Missing celery workers For me it is consistently every 60 seconds. Here are logs from the last 10 minutes. I also set loglevel to DEBUG for each config file in /etc/default/pulp* and restarted the services, but I'm not seeing any DEBUG stuff in the logs. $ sudo grep missing /var/log/messages Nov 18 11:23:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-0@pulp01' has gone missing, removing from list of workers Nov 18 11:23:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-0@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:23:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-2@pulp01' has gone missing, removing from list of workers Nov 18 11:23:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-2@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:23:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-3@pulp01' has gone missing, removing from list of workers Nov 18 11:23:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-3@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:23:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-1@pulp01' has gone missing, removing from list of workers Nov 18 11:23:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-1@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:23:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'resource_manager@pulp01' has gone missing, removing from list of workers Nov 18 11:23:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named resource_manager@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:24:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-2@pulp01' has gone missing, removing from list of workers Nov 18 11:24:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-2@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:24:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-0@pulp01' has gone missing, removing from list of workers Nov 18 11:24:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-0@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:24:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-3@pulp01' has gone missing, removing from list of workers Nov 18 11:24:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-3@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:24:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-1@pulp01' has gone missing, removing from list of workers Nov 18 11:24:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-1@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:24:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'resource_manager@pulp01' has gone missing, removing from list of workers Nov 18 11:24:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named resource_manager@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:25:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'resource_manager@pulp01' has gone missing, removing from list of workers Nov 18 11:25:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named resource_manager@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:25:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-0@pulp01' has gone missing, removing from list of workers Nov 18 11:25:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-0@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:25:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-1@pulp01' has gone missing, removing from list of workers Nov 18 11:25:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-1@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:26:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-0@pulp01' has gone missing, removing from list of workers Nov 18 11:26:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-0@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:26:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-2@pulp01' has gone missing, removing from list of workers Nov 18 11:26:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-2@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:26:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-1@pulp01' has gone missing, removing from list of workers Nov 18 11:26:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-1@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:26:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-3@pulp01' has gone missing, removing from list of workers Nov 18 11:26:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-3@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:26:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'resource_manager@pulp01' has gone missing, removing from list of workers Nov 18 11:26:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named resource_manager@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:27:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-3@pulp01' has gone missing, removing from list of workers Nov 18 11:27:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-3@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:27:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-0@pulp01' has gone missing, removing from list of workers Nov 18 11:27:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-0@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:27:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-2@pulp01' has gone missing, removing from list of workers Nov 18 11:27:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-2@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:27:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-1@pulp01' has gone missing, removing from list of workers Nov 18 11:27:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-1@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:27:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'resource_manager@pulp01' has gone missing, removing from list of workers Nov 18 11:27:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named resource_manager@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:28:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-3@pulp01' has gone missing, removing from list of workers Nov 18 11:28:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-3@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:28:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-0@pulp01' has gone missing, removing from list of workers Nov 18 11:28:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-0@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:28:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-2@pulp01' has gone missing, removing from list of workers Nov 18 11:28:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-2@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:28:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-1@pulp01' has gone missing, removing from list of workers Nov 18 11:28:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-1@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:28:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'resource_manager@pulp01' has gone missing, removing from list of workers Nov 18 11:28:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named resource_manager@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:29:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-0@pulp01' has gone missing, removing from list of workers Nov 18 11:29:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-0@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:29:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-2@pulp01' has gone missing, removing from list of workers Nov 18 11:29:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-2@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:29:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-1@pulp01' has gone missing, removing from list of workers Nov 18 11:29:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-1@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:29:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-3@pulp01' has gone missing, removing from list of workers Nov 18 11:29:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-3@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:29:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'resource_manager@pulp01' has gone missing, removing from list of workers Nov 18 11:29:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named resource_manager@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:30:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-0@pulp01' has gone missing, removing from list of workers Nov 18 11:30:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-0@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:30:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-2@pulp01' has gone missing, removing from list of workers Nov 18 11:30:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-2@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:30:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-1@pulp01' has gone missing, removing from list of workers Nov 18 11:30:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-1@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:30:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-3@pulp01' has gone missing, removing from list of workers Nov 18 11:30:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-3@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:30:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'resource_manager@pulp01' has gone missing, removing from list of workers Nov 18 11:30:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named resource_manager@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:31:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-0@pulp01' has gone missing, removing from list of workers Nov 18 11:31:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-0@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:31:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-2@pulp01' has gone missing, removing from list of workers Nov 18 11:31:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-2@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:31:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-1@pulp01' has gone missing, removing from list of workers Nov 18 11:31:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-1@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:31:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-3@pulp01' has gone missing, removing from list of workers Nov 18 11:31:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-3@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:31:34 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'resource_manager@pulp01' has gone missing, removing from list of workers Nov 18 11:31:34 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named resource_manager@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:32:35 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-0@pulp01' has gone missing, removing from list of workers Nov 18 11:32:35 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-0@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:32:35 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-2@pulp01' has gone missing, removing from list of workers Nov 18 11:32:35 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-2@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:32:35 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-1@pulp01' has gone missing, removing from list of workers Nov 18 11:32:35 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-1@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:32:35 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'reserved_resource_worker-3@pulp01' has gone missing, removing from list of workers Nov 18 11:32:35 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named reserved_resource_worker-3@pulp01 is missing. Canceling the tasks in its queue. Nov 18 11:32:35 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker 'resource_manager@pulp01' has gone missing, removing from list of workers Nov 18 11:32:35 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker named resource_manager@pulp01 is missing. Canceling the tasks in its queue. -----Original Message----- From: <a href="mailto:pulp-list-bounces@redhat.com">pulp-list-bounces@redhat.com</a> [mailto:<a href="mailto:pulp-list-bounces@redhat.com">pulp-list-bounces@redhat.com</a>] On Behalf Of Brian Bouterse Sent: Wednesday, November 18, 2015 11:05 AM To: <a href="mailto:pulp-list@redhat.com">pulp-list@redhat.com</a> Subject: Re: [Pulp-list] Missing celery workers Jason and Jeffrey, Thanks for reporting this. I've written up a bug [0] and I am investigating the root cause. On the bug are you able to leave some answers to these questions? - Can you confirm that it affects both RabbitMQ and Qpid usage? - Can you confirm that the workers "go missing" and then return, and then "go missing" in a continuous cycle? I expect it to happen every 90 seconds. - Jeffrey specifically, what OS are you using? [0]: <a href="https://pulp.plan.io/issues/1380" target="_blank">https://pulp.plan.io/issues/1380</a> Thanks, Brian On 11/18/2015 09:33 AM, Miller, Jeffrey L wrote: > I am seeing this behavior as well after upgrading from 2.6 to 2.7. > However, I am using qpid not rabbitmq. > > > > -Jeffrey > > > > > > > > *From:* <a href="mailto:pulp-list-bounces@redhat.com">pulp-list-bounces@redhat.com</a> > [mailto:<a href="mailto:pulp-list-bounces@redhat.com">pulp-list-bounces@redhat.com</a>] *On Behalf Of *Ashby, Jason > (IMS) > *Sent:* Wednesday, November 18, 2015 8:29 AM > *To:* <a href="mailto:pulp-list@redhat.com">pulp-list@redhat.com</a> > *Subject:* [Pulp-list] Missing celery workers > > > > Hi all, > > I'm hitting another issue with the upgrade to Pulp 2.7.0 + changing > from qpid to rabbitmq for messaging. The workers are continuously > going missing, every minute or so. The effect is that the tasks in > the task list stay in a Waiting state and are never completed. > > > > Rabbitmq looks healthy; I see successful accepted connections per the > logs and can see a bunch of connections in the rabbitmq management GUI. > I'm kind of stuck as far as troubleshooting goes. Any tips on what > else to investigate? > > > > Pulp and rabbitmq servers are both CentOS 6. > > > > # /var/log/messages > > Nov 18 08:53:56 pulp01 pulp: celery.worker.consumer:INFO: missed > heartbeat from resource_manager@pulp01 > > Nov 18 09:05:46 pulp01 pulp: pulp.server.async.worker_watcher:INFO: > New worker 'reserved_resource_worker-3@pulp01' discovered > > Nov 18 09:05:46 pulp01 pulp: pulp.server.async.worker_watcher:INFO: > New worker 'reserved_resource_worker-1@pulp01' discovered > > Nov 18 09:05:46 pulp01 pulp: pulp.server.async.worker_watcher:INFO: > New worker 'reserved_resource_worker-2@pulp01' discovered > > Nov 18 09:05:46 pulp01 pulp: pulp.server.async.worker_watcher:INFO: > New worker 'reserved_resource_worker-0@pulp01' discovered > > Nov 18 09:05:56 pulp01 pulp: pulp.server.async.worker_watcher:INFO: > New worker 'resource_manager@pulp01' discovered > > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker > 'reserved_resource_worker-3@pulp01' has gone missing, removing from > list of work > > ers > > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker > named reserved_resource_worker-3@pulp01 is missing. Canceling the > tasks in its q > > ueue. > > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker > 'reserved_resource_worker-1@pulp01' has gone missing, removing from > list of work > > ers > > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker > named reserved_resource_worker-1@pulp01 is missing. Canceling the > tasks in its q > > ueue. > > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker > 'reserved_resource_worker-2@pulp01' has gone missing, removing from > list of work > > ers > > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker > named reserved_resource_worker-2@pulp01 is missing. Canceling the > tasks in its q > > ueue. > > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker > 'reserved_resource_worker-0@pulp01' has gone missing, removing from > list of work > > ers > > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker > named reserved_resource_worker-0@pulp01 is missing. Canceling the > tasks in its q > > ueue. > > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.scheduler:ERROR: Worker > 'resource_manager@pulp01' has gone missing, removing from list of > workers > > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.tasks:ERROR: The worker > named resource_manager@pulp01 is missing. Canceling the tasks in its queue. > > Nov 18 09:06:46 pulp01 pulp: pulp.server.async.scheduler:ERROR: There > are 0 pulp_resource_manager processes running. Pulp will not operate > correctly without > > at least one pulp_resource_mananger process running. > > > > ---------------------------------------------------------------------- > -- > > > Information in this e-mail may be confidential. It is intended only > for the addressee(s) identified above. If you are not the > addressee(s), or an employee or agent of the addressee(s), please note > that any dissemination, distribution, or copying of this communication > is strictly prohibited. If you have received this e-mail in error, > please notify the sender of the error. > > > > _______________________________________________ > Pulp-list mailing list > <a href="mailto:Pulp-list@redhat.com">Pulp-list@redhat.com</a> > <a href="https://www.redhat.com/mailman/listinfo/pulp-list" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a> > _______________________________________________ Pulp-list mailing list <a href="mailto:Pulp-list@redhat.com">Pulp-list@redhat.com</a> <a href="https://www.redhat.com/mailman/listinfo/pulp-list" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a> ________________________________ Information in this e-mail may be confidential. It is intended only for the addressee(s) identified above. If you are not the addressee(s), or an employee or agent of the addressee(s), please note that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this e-mail in error, please notify the sender of the error. _______________________________________________ Pulp-list mailing list <a href="mailto:Pulp-list@redhat.com">Pulp-list@redhat.com</a> <a href="https://www.redhat.com/mailman/listinfo/pulp-list" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a> _______________________________________________ Pulp-list mailing list <a href="mailto:Pulp-list@redhat.com">Pulp-list@redhat.com</a> <a href="https://www.redhat.com/mailman/listinfo/pulp-list" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a><o:p></o:p> </div> </div> </blockquote> </div> <o:p> </o:p> </div> </div> <hr> Information in this e-mail may be confidential. It is intended only for the addressee(s) identified above. If you are not the addressee(s), or an employee or agent of the addressee(s), please note that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this e-mail in error, please notify the sender of the error. </body> </html>