<div dir="ltr"><div class="gmail_extra"><div><div class="gmail_signature"><div dir="ltr">Thank you very much Brian. Updating python-kombu to 3.0.33-5 resolves the issue. After the update I ran sync task several times without any issues. Please find the log as requested. # rpm -qa | grep kombupython-kombu-3.0.33-4.pulp.el7.noarch # sudo qpid-stat -q |grep celeryev celeryev.136f85c7-b6fd-4e90-8426-fc73ff86864c Y 0 2.27k 2.27k 0 1.92m 1.92m 1 2# journalctl -f -l-- Logs begin at Wed 2016-05-04 11:41:21 CEST. --May 04 15:21:03 mysrv audispd[975]: node=mysrv type=USER_ACCT msg=audit(1462368063.769:541): pid=3501 uid=0 auid=692982 ses=7 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:accounting grantors=pam_succeed_if acct="root" exe="/usr/bin/su" hostname=? addr=? terminal=pts/0 res=success'May 04 15:21:03 mysrv audispd[975]: node=mysrv type=CRED_ACQ msg=audit(1462368063.770:542): pid=3501 uid=0 auid=692982 ses=7 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_rootok acct="root" exe="/usr/bin/su" hostname=? addr=? terminal=pts/0 res=success'May 04 15:21:03 mysrv su[3501]: pam_unix(su-l:session): session opened for user root by lsagy92iy(uid=0)May 04 15:21:03 mysrv audispd[975]: node=mysrv type=USER_START msg=audit(1462368063.775:543): pid=3501 uid=0 auid=692982 ses=7 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:session_open grantors=pam_keyinit,pam_keyinit,pam_limits,pam_systemd,pam_unix,pam_xauth acct="root" exe="/usr/bin/su" hostname=? addr=? terminal=pts/0 res=success'May 04 15:22:21 mysrv sudo[3536]: root : TTY=pts/0 ; PWD=/root ; USER=root ; COMMAND=/bin/qpid-stat -qMay 04 15:22:21 mysrv audispd[975]: node=mysrv type=USER_CMD msg=audit(1462368141.089:544): pid=3536 uid=0 auid=692982 ses=7 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='cwd="/root" cmd=717069642D73746174202D71 terminal=pts/0 res=success'May 04 15:22:21 mysrv audispd[975]: node=mysrv type=CRED_ACQ msg=audit(1462368141.090:545): pid=3536 uid=0 auid=692982 ses=7 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_env,pam_localuser,pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'May 04 15:22:21 mysrv audispd[975]: node=mysrv type=USER_START msg=audit(1462368141.090:546): pid=3536 uid=0 auid=692982 ses=7 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:session_open grantors=pam_keyinit,pam_limits acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'May 04 15:22:21 mysrv audispd[975]: node=mysrv type=USER_END msg=audit(1462368141.240:547): pid=3536 uid=0 auid=692982 ses=7 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:session_close grantors=pam_keyinit,pam_limits acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'May 04 15:22:21 mysrv audispd[975]: node=mysrv type=CRED_DISP msg=audit(1462368141.240:548): pid=3536 uid=0 auid=692982 ses=7 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_env,pam_localuser,pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success' ^C# rpm -Uvh python-kombu-3.0.33-5.pulp.el7.noarch.rpmPreparing... ################################# [100%]Updating / installing... 1:python-kombu-1:3.0.33-5.pulp.el7 ################################# [ 50%]Cleaning up / removing... 2:python-kombu-1:3.0.33-4.pulp.el7 ################################# [100%]# rpm -qa | grep kombupython-kombu-3.0.33-5.pulp.el7.noarch # sudo qpid-stat -q |grep celeryev celeryev.a2ebe0b7-c01e-41f2-8fae-58498cee82e5 Y 0 109 109 0 91.6k 91.6k 1 2 # pulp-admin status+----------------------------------------------------------------------+ Status of the server+----------------------------------------------------------------------+ Api Version: 2Database Connection: Connected: TrueKnown Workers: _id: scheduler@mysrv _ns: workers Last Heartbeat: 2016-05-04T13:47:15Z _id: reserved_resource_worker-3@mysrv _ns: workers Last Heartbeat: 2016-05-04T13:48:15Z _id: reserved_resource_worker-0@mysrv _ns: workers Last Heartbeat: 2016-05-04T13:48:15Z _id: reserved_resource_worker-2@mysrv _ns: workers Last Heartbeat: 2016-05-04T13:48:15Z _id: reserved_resource_worker-1@mysrv _ns: workers Last Heartbeat: 2016-05-04T13:48:18Z _id: resource_manager@mysrv _ns: workers Last Heartbeat: 2016-05-04T13:48:18ZMessaging Connection: Connected: TrueVersions: Platform Version: 2.8.2 # qpid-stat -qQueues queue dur autoDel excl msg msgIn msgOut bytes bytesIn bytesOut cons bind ============================================================================================= ======================================================= 05a8bc3d-cf7e-4fae-875b-13b214cf8de6:1.0 Y Y 0 0 0 0 0 0 1 2 21c58b6c-8bd4-4ede-86d9-fd65dfe8bc4c:1.0 Y Y 0 8 8 0 4.90k 4.90k 1 2 21c58b6c-8bd4-4ede-86d9-fd65dfe8bc4c:2.0 Y Y 0 4 4 0 2.50k 2.50k 1 2 372baf00-05a0-40e0-bd16-430433ce0980:1.0 Y Y 0 2 2 0 486 486 1 2 39ae3a61-e29e-4032-af1e-19bd822e34e3:1.0 Y Y 0 8 8 0 4.88k 4.88k 1 2 39ae3a61-e29e-4032-af1e-19bd822e34e3:2.0 Y Y 0 4 4 0 2.52k 2.52k 1 2 3a7d8a98-86df-471d-9e20-384dd8dacbb2:1.0 Y Y 0 2 2 0 486 486 1 2 4fa605d0-5346-4087-852d-266f98d708e4:0.0 Y Y 0 0 0 0 0 0 1 2 96497b3d-6fdc-46c7-b784-c3508f97500b:1.0 Y Y 0 8 8 0 4.88k 4.88k 1 2 96497b3d-6fdc-46c7-b784-c3508f97500b:2.0 Y Y 0 4 4 0 2.52k 2.52k 1 2 a2a2aaee-797c-4b47-853d-d60b80bfac69:1.0 Y Y 0 5 5 0 2.73k 2.73k 1 2 b1918f13-ff85-45ec-91a7-67dceea00fb2:1.0 Y Y 0 2 2 0 486 486 1 2 c209635c-dba5-43b5-bc7e-284c342e0ed0:1.0 Y Y 0 8 8 0 4.88k 4.88k 1 2 c209635c-dba5-43b5-bc7e-284c342e0ed0:2.0 Y Y 0 4 4 0 2.46k 2.46k 1 2 celery Y 0 0 0 0 0 0 4 2 celeryev.a2ebe0b7-c01e-41f2-8fae-58498cee82e5 Y 0 144 144 0 121k 121k 1 2 d78d70a1-7ecd-4375-8a3a-ad7de2f60fa3:1.0 Y Y 0 8 8 0 4.88k 4.88k 1 2 d78d70a1-7ecd-4375-8a3a-ad7de2f60fa3:2.0 Y Y 0 4 4 0 2.52k 2.52k 1 2 f2fea2ac-4468-4e4a-9843-86cfab4157f2:1.0 Y Y 0 2 2 0 486 486 1 2 f9e48a42-8b2c-4378-b90a-1632ff8dbba8:1.0 Y Y 0 2 2 0 486 486 1 2 pulp.task Y 0 0 0 0 0 0 3 1 <a href="mailto:reserved_resource_worker-0@mysrv.celery.pidbox">reserved_resource_worker-0@mysrv.celery.pidbox</a> Y 0 1 1 0 449 449 1 2 <a href="mailto:reserved_resource_worker-0@mysrv.dq">reserved_resource_worker-0@mysrv.dq</a> Y Y 0 0 0 0 0 0 1 2 <a href="mailto:reserved_resource_worker-1@mysrv.celery.pidbox">reserved_resource_worker-1@mysrv.celery.pidbox</a> Y 0 1 1 0 449 449 1 2 <a href="mailto:reserved_resource_worker-1@mysrv.dq">reserved_resource_worker-1@mysrv.dq</a> Y Y 0 0 0 0 0 0 1 2 <a href="mailto:reserved_resource_worker-2@mysrv.celery.pidbox">reserved_resource_worker-2@mysrv.celery.pidbox</a> Y 0 1 1 0 449 449 1 2 <a href="mailto:reserved_resource_worker-2@mysrv.dq">reserved_resource_worker-2@mysrv.dq</a> Y Y 0 0 0 0 0 0 1 2 <a href="mailto:reserved_resource_worker-3@mysrv.celery.pidbox">reserved_resource_worker-3@mysrv.celery.pidbox</a> Y 0 1 1 0 449 449 1 2 <a href="mailto:reserved_resource_worker-3@mysrv.dq">reserved_resource_worker-3@mysrv.dq</a> Y Y 0 0 0 0 0 0 1 2 resource_manager Y 0 0 0 0 0 0 1 2 <a href="mailto:resource_manager@mysrv.celery.pidbox">resource_manager@mysrv.celery.pidbox</a> Y 0 0 0 0 0 0 1 2 <a href="mailto:resource_manager@mysrv.dq">resource_manager@mysrv.dq</a> Y Y 0 0 0 0 0 0 1 2 # pulp-admin -vv tasks details --task-id bf077e15-09f0-45d2-98d7-17ce953248d1+----------------------------------------------------------------------+ Task Details+----------------------------------------------------------------------+ 2016-05-04 15:59:19,827 - DEBUG - sending GET request to /pulp/api/v2/tasks/bf077e15-09f0-45d2-98d7-17ce953248d1/2016-05-04 15:59:19,950 - INFO - GET request to /pulp/api/v2/tasks/bf077e15-09f0-45d2-98d7-17ce953248d1/ with parameters None2016-05-04 15:59:19,950 - INFO - Response status : 200 2016-05-04 15:59:19,950 - INFO - Response body :{ "exception": null, "task_type": "pulp.server.managers.repo.sync.sync", "_href": "/pulp/api/v2/tasks/bf077e15-09f0-45d2-98d7-17ce953248d1/", "task_id": "bf077e15-09f0-45d2-98d7-17ce953248d1", "tags": [ "pulp:repository:rhel-6-server-rpms", "pulp:action:sync" ], "finish_time": null, "_ns": "task_status", "start_time": "2016-05-04T13:51:26Z", "traceback": null, "spawned_tasks": [], "progress_report": { "yum_importer": { "content": { "size_total": 0, "items_left": 0, "items_total": 0, "state": "IN_PROGRESS", "size_left": 0, "details": { "rpm_total": 0, "rpm_done": 0, "drpm_total": 0, "drpm_done": 0 }, "error_details": [] }, "comps": { "state": "NOT_STARTED" }, "purge_duplicates": { "state": "NOT_STARTED" }, "distribution": { "items_total": 0, "state": "NOT_STARTED", "error_details": [], "items_left": 0 }, "errata": { "state": "NOT_STARTED" }, "metadata": { "state": "FINISHED" } } }, "queue": "<a href="mailto:reserved_resource_worker-2@mysrv.dq">reserved_resource_worker-2@mysrv.dq</a>", "state": "running", "worker_name": "reserved_resource_worker-2@mysrv", "result": null, "error": null, "_id": { "$oid": "5729fe5e89348df3077e3e52" }, "id": "5729fe5e89348df3077e3e52"} Operations: syncResources: rhel-6-server-rpms (repository)State: RunningStart Time: 2016-05-04T13:51:26ZFinish Time: IncompleteResult: IncompleteTask Id: bf077e15-09f0-45d2-98d7-17ce953248d1Progress Report: Yum Importer: Comps: State: NOT_STARTED Content: Details: Drpm Done: 0 Drpm Total: 0 Rpm Done: 0 Rpm Total: 0 Error Details: Items Left: 0 Items Total: 0 Size Left: 0 Size Total: 0 State: IN_PROGRESS Distribution: Error Details: Items Left: 0 Items Total: 0 State: NOT_STARTED Errata: State: NOT_STARTED Metadata: State: FINISHED Purge Duplicates: State: NOT_STARTED Thanks,Samiron</div></div></div> <div class="gmail_quote">On Wed, May 4, 2016 at 6:05 PM, Brian Bouterse <<a href="mailto:bbouters@redhat.com" target="_blank">bbouters@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">After trying the things from my other e-mail. When your system is in its "bad state" show the `qpid-stat -q` output too please. The `qpid-stat -q` output from this e-mail shows all of the queues are empty with msgIn and msgOut being the same. -Brian On 05/04/2016 05:07 AM, Mallick, Samiron wrote: > *Seems the queue has something. I deleted them up one-by-one, and tried > starting the stuck repo sync. and it works for the repo finally. But bad > news is, the queue again regenerates lot of tasks automatically where no > sync tasks running. So while trying to run another repo, it went to > Waiting again. Is there any workaround yet?* <div><div class="h5">> > > > # qpid-stat -q > > Queues > > queue > dur autoDel excl msg msgIn msgOut bytes bytesIn bytesOut cons > bind > > > ==================================================================================================================================================== > > > 1a4b6e57-3ecc-406d-84cd-29b24a0a6610:1.0 Y > Y 0 2 2 0 486 486 1 2 > > > 36e7ca4b-5a0d-4f5c-9f94-a22016390562:1.0 Y > Y 0 8 8 0 4.91k 4.91k 1 2 > > > 36e7ca4b-5a0d-4f5c-9f94-a22016390562:2.0 Y > Y 0 4 4 0 2.50k 2.50k 1 2 > > > 3de2643d-bb8d-4e98-94d1-d8ed4e1bdf11:1.0 Y > Y 0 8 8 0 4.88k 4.88k 1 2 > > > 3de2643d-bb8d-4e98-94d1-d8ed4e1bdf11:2.0 Y > Y 0 4 4 0 2.52k 2.52k 1 2 > > > 43099b2b-cc78-4b96-a1a9-50d94517c1e2:1.0 Y > Y 0 2 2 0 486 486 1 2 > > > 4409c371-0d54-44c4-94b7-ec0bb7ecfd45:1.0 Y > Y 0 0 0 0 0 0 1 2 > > 680eb17a-8285-450c-b8b9-51d107b4ff2d:0.0 > Y Y 0 0 0 0 0 > 0 1 2 > > > bcbc1fa3-8157-403d-8f33-252fe057587a:1.0 Y > Y 0 5 5 0 2.67k 2.67k 1 2 > > celery > Y > 0 0 0 0 0 0 1 2 > > > celeryev.4021d653-24bf-4f06-9aee-aa457c579c4b Y > 0 12 12 0 10.0k 10.0k 1 2 > > pulp.task > Y 0 0 0 0 0 0 > 3 1 > > reserved_resource_worker-0@mysrv.celery.pidbox </div></div>> <mailto:<a href="mailto:reserved_resource_worker-0@mysrv.celery.pidbox">reserved_resource_worker-0@mysrv.celery.pidbox</a>> > Y 0 0 0 0 0 0 1 2 > > reserved_resource_worker-0@mysrv.dq > <mailto:<a href="mailto:reserved_resource_worker-0@mysrv.dq">reserved_resource_worker-0@mysrv.dq</a>> Y > Y 0 0 0 0 0 0 1 2 > > resource_manager > Y 0 0 0 0 0 0 > 1 2 > > resource_manager@mysrv.celery.pidbox > <mailto:<a href="mailto:resource_manager@mysrv.celery.pidbox">resource_manager@mysrv.celery.pidbox</a>> > Y 0 0 0 0 0 0 1 2 > > resource_manager@mysrv.dq > <mailto:<a href="mailto:resource_manager@mysrv.dq">resource_manager@mysrv.dq</a>> Y <div><div class="h5">> Y 0 0 0 0 0 0 1 2 > > # pulp-admin tasks list > > +----------------------------------------------------------------------+ > > Tasks > > +----------------------------------------------------------------------+ > > > > No tasks found > > > > > > > > # qpid-tool > > Management Tool for QPID > > qpid: list > > Summary of Objects by Type: > > qpid: help > > Management Tool for QPID > > > > Commands: > > agents - Print a list of the known Agents > > list - Print summary of existing objects > by class > > list <className> - Print list of objects of the > specified class > > list <className> active - Print list of non-deleted objects > of the specified class > > show <ID> - Print contents of an object (infer > className) > > call <ID> <methodName> [<args>] - Invoke a method on an object > > schema - Print summary of object classes > seen on the target > > schema <className> - Print details of an object class > > set time-format short - Select short timestamp format > (default) > > set time-format long - Select long timestamp format > > quit or ^D - Exit the program > > > > qpid: list > > Summary of Objects by Type: > > Package Class Active Deleted > > ============================================================ > > org.apache.qpid.broker binding 43 12 > > org.apache.qpid.broker broker 1 0 > > org.apache.qpid.broker memory 1 0 > > org.apache.qpid.broker system 1 0 > > org.apache.qpid.linearstore store 1 0 > > org.apache.qpid.broker subscription 23 5 > > org.apache.qpid.broker connection 14 1 > > org.apache.qpid.broker session 19 1 > > org.apache.qpid.linearstore journal 5 0 > > org.apache.qpid.acl acl 1 0 > > org.apache.qpid.broker queue 21 5 > > org.apache.qpid.broker exchange 13 0 > > org.apache.qpid.broker vhost 1 0 > > qpid: list queue > > Object Summary: > > ID Created Destroyed Index > > > ============================================================================================================================ > > 114 06:24:09 06:24:41 > org.apache.qpid.broker:queue:topic-mysrv.3108.1 > > 115 06:24:09 06:24:41 > org.apache.qpid.broker:queue:reply-mysrv.3108.1 > > 116 06:24:09 06:24:41 > org.apache.qpid.broker:queue:qmfc-v2-ui-mysrv.3108.1 > > 117 06:24:09 06:24:41 > org.apache.qpid.broker:queue:qmfc-v2-mysrv.3108.1 > > 118 06:24:09 06:24:41 > org.apache.qpid.broker:queue:qmfc-v2-hb-mysrv.3108.1 > > 198 06:16:36 - > org.apache.qpid.broker:queue:1a4b6e57-3ecc-406d-84cd-29b24a0a6610:1.0 > > 199 06:16:36 - > org.apache.qpid.broker:queue:36e7ca4b-5a0d-4f5c-9f94-a22016390562:1.0 > > 200 06:16:38 - > org.apache.qpid.broker:queue:36e7ca4b-5a0d-4f5c-9f94-a22016390562:2.0 > > 201 06:16:36 - > org.apache.qpid.broker:queue:3de2643d-bb8d-4e98-94d1-d8ed4e1bdf11:1.0 > > 202 06:16:37 - > org.apache.qpid.broker:queue:3de2643d-bb8d-4e98-94d1-d8ed4e1bdf11:2.0 > > 203 06:16:36 - > org.apache.qpid.broker:queue:43099b2b-cc78-4b96-a1a9-50d94517c1e2:1.0 > > 204 06:16:33 - > org.apache.qpid.broker:queue:4409c371-0d54-44c4-94b7-ec0bb7ecfd45:1.0 > > 205 06:16:33 - > org.apache.qpid.broker:queue:bcbc1fa3-8157-403d-8f33-252fe057587a:1.0 > > 206 06:16:33 - org.apache.qpid.broker:queue:celery > > 207 06:16:33 - > org.apache.qpid.broker:queue:celeryev.4021d653-24bf-4f06-9aee-aa457c579c4b > > 208 06:16:33 - org.apache.qpid.broker:queue:pulp.task > > 209 06:24:43 - > org.apache.qpid.broker:queue:qmfc-v2-hb-mysrv.3122.1 > > 210 06:24:43 - > org.apache.qpid.broker:queue:qmfc-v2-mysrv.3122.1 > > 211 06:24:43 - > org.apache.qpid.broker:queue:qmfc-v2-ui-mysrv.3122.1 > > 212 06:24:43 - > org.apache.qpid.broker:queue:reply-mysrv.3122.1 > > 213 06:16:37 - > org.apache.qpid.broker:queue:reserved_resource_worker-0@mysrv.celery.pidbox > > 214 06:16:36 - > org.apache.qpid.broker:queue:reserved_resource_worker-0@mysrv.dq > > 215 06:16:33 - org.apache.qpid.broker:queue:resource_manager > > 216 06:16:38 - > org.apache.qpid.broker:queue:resource_manager@mysrv.celery.pidbox > > 217 06:16:37 - > org.apache.qpid.broker:queue:resource_manager@mysrv.dq > > 218 06:24:43 - > org.apache.qpid.broker:queue:topic-mysrv.3122.1 > > > > # qpid-config del queue 1a4b6e57-3ecc-406d-84cd-29b24a0a6610:1.0 > > Failed: Exception: Exception from Agent: {u'error_code': 7, > u'error_text': 'precondition-failed: Cannot delete queue > 1a4b6e57-3ecc-406d-84cd-29b24a0a6610:1.0; queue in use > (/builddir/build/BUILD/qpid-cpp-0.34/src/qpid/broker/Broker.cpp:1068)'} > > > > # qpid-config del queue resource_manager@mysrv.dq </div></div>> <mailto:<a href="mailto:resource_manager@mysrv.dq">resource_manager@mysrv.dq</a>> --force <div><div class="h5">> > > > qpid: list > > Summary of Objects by Type: > > Package Class Active Deleted > > ============================================================ > > org.apache.qpid.broker binding 16 0 > > org.apache.qpid.broker broker 1 0 > > org.apache.qpid.broker memory 1 0 > > org.apache.qpid.broker system 1 0 > > org.apache.qpid.linearstore store 1 0 > > org.apache.qpid.broker subscription 7 0 > > org.apache.qpid.broker connection 13 0 > > org.apache.qpid.broker session 269 0 > > org.apache.qpid.acl acl 1 0 > > org.apache.qpid.broker queue 7 0 > > org.apache.qpid.broker exchange 13 0 > > org.apache.qpid.broker vhost 1 0 > > qpid: list queue > > Object Summary: > > ID Created Destroyed Index > > > ====================================================================================================== > > 146 08:47:30 - > org.apache.qpid.broker:queue:2d1a7c8f-bc3b-4d54-bbe6-b7b264530506:1.0 > > 147 08:47:30 - > org.apache.qpid.broker:queue:celeryev.d45c6bc2-2449-4700-b3bb-bbbbf0b2990b > > 148 08:52:24 - > org.apache.qpid.broker:queue:qmfc-v2-hb-mysrv.4080.1 > > 149 08:52:24 - > org.apache.qpid.broker:queue:qmfc-v2-mysrv.4080.1 > > 150 08:52:24 - > org.apache.qpid.broker:queue:qmfc-v2-ui-mysrv.4080.1 > > 151 08:52:24 - > org.apache.qpid.broker:queue:reply-mysrv.4080.1 > > 152 08:52:24 - > org.apache.qpid.broker:queue:topic-mysrv.4080.1 > > > > # pulp-admin tasks list > > +----------------------------------------------------------------------+ > > Tasks > > +----------------------------------------------------------------------+ > > > > No tasks found > > > > # pulp-admin rpm repo sync run --repo-id=rhel-6-server-rpms > > +----------------------------------------------------------------------+ > > Synchronizing Repository [rhel-6-server-rpms] > > +----------------------------------------------------------------------+ > > > > This command may be exited via ctrl+c without affecting the request. > > > > > > [\] > > *** STUCK *** > > > > On Wed, May 4, 2016 at 9:37 AM, Mallick, Samiron </div></div>> <<a href="mailto:samiron.mallick@gmail.com">samiron.mallick@gmail.com</a> <mailto:<a href="mailto:samiron.mallick@gmail.com">samiron.mallick@gmail.com</a>>> wrote: > > Hey Brian, thanks for the reply. > > *From the below output I could see "resource_worker-1" is > responsible for this task and I have 4 worker displayed on the > server.____* > > __ __ > > # pulp-admin tasks list____ > > +----------------------------------------------------------------------+____ > > Tasks____ > > +----------------------------------------------------------------------+____ > > __ __ > > Operations: sync____ > > Resources: rhel-6-server-supplementary-rpms (repository)____ > > State: Waiting____ > > Start Time: Unstarted____ > > Finish Time: Incomplete____ > > Task Id: 49b83f70-e6d6-4cdb-9c5a-93c20c31d697____ > > __ __ > > __ __ > > # pulp-admin -vv tasks details --task-id > 49b83f70-e6d6-4cdb-9c5a-93c20c31d697____ > > +----------------------------------------------------------------------+____ > > Task Details____ > > +----------------------------------------------------------------------+____ > > __ __ > > 2016-05-04 04:55:33,231 - DEBUG - sending GET request to > /pulp/api/v2/tasks/49b83f70-e6d6-4cdb-9c5a-93c20c31d697/____ > > 2016-05-04 04:55:33,362 - INFO - GET request to > /pulp/api/v2/tasks/49b83f70-e6d6-4cdb-9c5a-93c20c31d697/ with > parameters None____ <div><div class="h5">> > 2016-05-04 04:55:33,362 - INFO - Response status : 200____ > > __ __ > > 2016-05-04 04:55:33,363 - INFO - Response body :____ > > {____ > > "exception": null,____ > > "task_type": "pulp.server.managers.repo.sync.sync",____ > > "_href": > "/pulp/api/v2/tasks/49b83f70-e6d6-4cdb-9c5a-93c20c31d697/",____ > > "task_id": "49b83f70-e6d6-4cdb-9c5a-93c20c31d697",____ > > "tags": [____ > > "pulp:repository:rhel-6-server-supplementary-rpms",____ > > "pulp:action:sync"____ > > ],____ > > "finish_time": null,____ > > "_ns": "task_status",____ > > "start_time": null,____ > > "traceback": null,____ > > "spawned_tasks": [],____ > > "progress_report": {},____ > > "queue": "reserved_resource_worker-1@mysrv.dq",____ > > "state": "waiting",____ > > "worker_name": "reserved_resource_worker-1@mysrv",____ > > "result": null,____ > > "error": null,____ > > "_id": {____ > > "$oid": "572964399b70a2ea1d2694aa"____ > > },____ > > "id": "572964399b70a2ea1d2694aa"____ > > }____ > > __ __ > > Operations: sync____ > > Resources: rhel-6-server-supplementary-rpms (repository)____ > > State: Waiting____ > > Start Time: Unstarted____ > > Finish Time: Incomplete____ > > Result: Incomplete____ > > Task Id: 49b83f70-e6d6-4cdb-9c5a-93c20c31d697____ > > Progress Report:____ > > __ __ > > __ __ > > # pulp-admin status____ > > +----------------------------------------------------------------------+____ > > Status of the server____ > > +----------------------------------------------------------------------+____ > > __ __ > > Api Version: 2____ > > Database Connection:____ > > Connected: True____ > > Known Workers:____ > > _id: scheduler@mysrv____ > > _ns: workers____ > > Last Heartbeat: 2016-05-04T02:53:34Z____ > > _id: reserved_resource_worker-3@mysrv____ > > _ns: workers____ > > Last Heartbeat: 2016-05-04T02:54:00Z____ > > _id: reserved_resource_worker-2@mysrv____ > > _ns: workers____ > > Last Heartbeat: 2016-05-04T02:54:00Z____ > > _id: resource_manager@mysrv____ > > _ns: workers____ > > Last Heartbeat: 2016-05-04T02:54:00Z____ > > _id: reserved_resource_worker-1@mysrv____ > > _ns: workers____ > > Last Heartbeat: 2016-05-04T02:54:01Z____ > > _id: reserved_resource_worker-0@mysrv____ > > _ns: workers____ > > Last Heartbeat: 2016-05-04T02:54:03Z____ > > Messaging Connection:____ > > Connected: True____ > > Versions:____ > > Platform Version: 2.8.2____ > > __ __ > > # ps -awfux | grep celery____ > </div></div>> root 4637 0.0 0.0 112644 960 pts/0 S+ 04:56 > 0:00 \_ grep --color=auto celery____ > > apache 1592 0.0 1.4 667716 56368 ? Ssl May03 0:26 > /usr/bin/python /usr/bin/celery worker -A pulp.server.async.app -n > resource_manager@%h -Q resource_manager -c 1 --events --umask 18 > --pidfile=/var/run/pulp/resource_manager.pid --heartbeat-interval=30____ > > apache 2921 0.0 1.4 667664 54296 ? Sl May03 0:13 \_ > /usr/bin/python /usr/bin/celery worker -A pulp.server.async.app -n > resource_manager@%h -Q resource_manager -c 1 --events --umask 18 > --pidfile=/var/run/pulp/resource_manager.pid --heartbeat-interval=30____ > > apache 1616 0.0 1.4 667996 56400 ? Ssl May03 0:27 > /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-0@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var/run/pulp/reserved_resource_worker-0.pid > --heartbeat-interval=30____ > > apache 2919 0.0 1.4 741536 54564 ? Sl May03 0:11 \_ > /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-0@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var/run/pulp/reserved_resource_worker-0.pid > --heartbeat-interval=30____ > > apache 1626 0.0 1.5 668560 59524 ? Ssl May03 0:29 > /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-1@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var/run/pulp/reserved_resource_worker-1.pid > --heartbeat-interval=30____ > > apache 4561 0.0 1.4 668560 56260 ? S 04:47 0:00 \_ > /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-1@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var/run/pulp/reserved_resource_worker-1.pid > --heartbeat-interval=30____ > > apache 1631 0.0 1.5 667748 58508 ? Ssl May03 0:27 > /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-2@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var/run/pulp/reserved_resource_worker-2.pid > --heartbeat-interval=30____ > > apache 2922 4.2 8.0 1042956 311476 ? Sl May03 48:25 \_ > /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-2@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var/run/pulp/reserved_resource_worker-2.pid > --heartbeat-interval=30____ > > apache 1637 0.0 1.4 667744 56368 ? Ssl May03 0:27 > /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-3@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var/run/pulp/reserved_resource_worker-3.pid > --heartbeat-interval=30____ > > apache 2920 0.0 1.4 815420 54760 ? Sl May03 0:13 \_ > /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-3@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var/run/pulp/reserved_resource_worker-3.pid > --heartbeat-interval=30____ > > apache 4620 6.5 0.8 663652 31432 ? Ssl 04:56 0:00 > /usr/bin/python /usr/bin/celery beat > --app=pulp.server.async.celery_instance.celery > --scheduler=pulp.server.async.scheduler.Scheduler____ > > __ __ > > __ __ > > *As I saw errors in output of pulp_worker-1, I restarted each worker > individually, and all error seems gone.____* > > __ __ > > # systemctl status pulp_workers.service____ > > ● pulp_workers.service - Pulp Celery Workers____ > > Loaded: loaded (/usr/lib/systemd/system/pulp_workers.service; > enabled; vendor preset: disabled)____ > > Active: active (exited) since Wed 2016-05-04 05:36:38 CEST; 3s > ago____ > > Process: 5717 ExecStop=/usr/bin/python -m > pulp.server.async.manage_workers stop (code=exited, > status=0/SUCCESS)____ > > Process: 5731 ExecStart=/usr/bin/python -m > pulp.server.async.manage_workers start (code=exited, > status=0/SUCCESS)____ > > Main PID: 5731 (code=exited, status=0/SUCCESS)____ > > __ __ > > May 04 05:36:38 mysrv systemd[1]: Starting Pulp Celery Workers...____ > > May 04 05:36:38 mysrv systemd[1]: Started Pulp Celery Workers.____ > > __ __ > > # systemctl status pulp_worker-0____ > > ? pulp_worker-0.service - Pulp Worker #0____ > > Loaded: loaded (/run/systemd/system/pulp_worker-0.service; > static; vendor preset: disabled)____ > > Active: active (running) since Wed 2016-05-04 05:10:44 CEST; 1min > 26s ago____ > > Main PID: 4753 (celery)____ > > CGroup: /system.slice/pulp_worker-0.service____ > > +-4753 /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-0@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var...____ > > +-4766 /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-0@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var...____ > > __ __ > > May 04 05:10:46 mysrv celery[4753]: - ** ---------- .> transport: > qpid://mysrv:5672// <<a href="http://srpadinf0244.mgt.insim.biz:5672//" rel="noreferrer" target="_blank">http://srpadinf0244.mgt.insim.biz:5672//</a>>____ > > May 04 05:10:46 mysrv celery[4753]: - ** ---------- .> results: > disabled____ > > May 04 05:10:46 mysrv celery[4753]: - *** --- * --- .> concurrency: > 1 (prefork)____ > > May 04 05:10:46 mysrv celery[4753]: -- ******* ----____ > > May 04 05:10:46 mysrv celery[4753]: --- ***** ----- [queues]____ > > May 04 05:10:46 mysrv celery[4753]: -------------- .> > celery exchange=celery(direct) key=celery____ > > May 04 05:10:46 mysrv celery[4753]: .> > reserved_resource_worker-0@mysrv.dq exchange=C.dq(direct) > key=rese...s <<a href="http://sim.biz/" rel="noreferrer" target="_blank">http://sim.biz/</a>>rv____ > > May 04 05:10:46 mysrv pulp[4753]: kombu.transport.qpid:INFO: > Connected to qpid with SASL mechanism ANONYMOUS____ > > May 04 05:10:46 mysrv pulp[4753]: celery.worker.consumer:INFO: > Connected to qpid://mysrv:5672// > <<a href="http://srpadinf0244.mgt.insim.biz:5672//" rel="noreferrer" target="_blank">http://srpadinf0244.mgt.insim.biz:5672//</a>>____ > > May 04 05:10:46 mysrv pulp[4753]: kombu.transport.qpid:INFO: > Connected to qpid with SASL mechanism ANONYMOUS____ > > Hint: Some lines were ellipsized, use -l to show in full.____ > > __ __ > > __ __ > > # systemctl status pulp_worker-1____ > > ? pulp_worker-1.service - Pulp Worker #1____ > > Loaded: loaded (/run/systemd/system/pulp_worker-1.service; > static; vendor preset: disabled)____ > > Active: active (running) since Wed 2016-05-04 05:08:16 CEST; 3min > 57s ago____ > > Main PID: 4718 (celery)____ > > CGroup: /system.slice/pulp_worker-1.service____ > > +-4718 /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-1@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var...____ > > +-4733 /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-1@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var...____ > > __ __ > > May 04 05:08:23 mysrv pulp[4718]: celery.worker.strategy:INFO: > Received task: > pulp.server.controllers.repository.download_def...3cc3c36]____ > > May 04 05:08:23 mysrv pulp[4718]: celery.worker.strategy:INFO: > Received task: > pulp.server.controllers.repository.download_def...ce7430b]____ > > May 04 05:08:23 mysrv pulp[4718]: celery.worker.job:INFO: Task > pulp.server.controllers.repository.download_deferred[aad88f32-...9s: > None____ > > May 04 05:08:23 mysrv pulp[4718]: celery.worker.strategy:INFO: > Received task: > pulp.server.db.reaper.reap_expired_documents[02...8322faa]____ > > May 04 05:08:23 mysrv pulp[4718]: celery.worker.strategy:INFO: > Received task: > pulp.server.controllers.repository.download_def...ddadf87]____ > > May 04 05:08:23 mysrv pulp[4718]: celery.worker.strategy:INFO: > Received task: > pulp.server.controllers.repository.download_def...d0cf8c6]____ > > May 04 05:08:23 mysrv pulp[4718]: celery.worker.strategy:INFO: > Received task: > pulp.server.controllers.repository.download_def...72edf98]____ > > May 04 05:08:23 mysrv pulp[4718]: celery.worker.strategy:INFO: > Received task: > pulp.server.controllers.repository.download_def...1e9e4bc]____ > > May 04 05:08:23 mysrv pulp[4718]: celery.worker.strategy:INFO: > Received task: > pulp.server.controllers.repository.download_def...30f8627]____ > > May 04 05:08:23 mysrv pulp[4718]: celery.worker.strategy:INFO: > Received task: > pulp.server.controllers.repository.queue_downlo...fd23e13]____ > > Hint: Some lines were ellipsized, use -l to show in full.____ > > __ __ > > __ __ > > # systemctl status pulp_worker-2____ > > ? pulp_worker-2.service - Pulp Worker #2____ > > Loaded: loaded (/run/systemd/system/pulp_worker-2.service; > static; vendor preset: disabled)____ > > Active: active (running) since Wed 2016-05-04 05:11:06 CEST; 1min > 10s ago____ > > Main PID: 4776 (celery)____ > > CGroup: /system.slice/pulp_worker-2.service____ > > +-4776 /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-2@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var...____ > > +-4789 /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-2@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var...____ > > __ __ > > May 04 05:11:07 mysrv celery[4776]: - ** ---------- .> transport: > qpid://mysrv:5672// <<a href="http://srpadinf0244.mgt.insim.biz:5672//" rel="noreferrer" target="_blank">http://srpadinf0244.mgt.insim.biz:5672//</a>>____ > > May 04 05:11:07 mysrv celery[4776]: - ** ---------- .> results: > disabled____ > > May 04 05:11:07 mysrv celery[4776]: - *** --- * --- .> concurrency: > 1 (prefork)____ > > May 04 05:11:07 mysrv celery[4776]: -- ******* ----____ > > May 04 05:11:07 mysrv celery[4776]: --- ***** ----- [queues]____ > > May 04 05:11:07 mysrv celery[4776]: -------------- .> > celery exchange=celery(direct) key=celery____ > > May 04 05:11:07 mysrv celery[4776]: .> > reserved_resource_worker-2@srpadinf0244.mgt.insim.biz.dq > exchange=C.dq(direct) key=rese...s <<a href="http://sim.biz/" rel="noreferrer" target="_blank">http://sim.biz/</a>>rv____ > > May 04 05:11:07 mysrv pulp[4776]: kombu.transport.qpid:INFO: > Connected to qpid with SASL mechanism ANONYMOUS____ > > May 04 05:11:07 mysrv pulp[4776]: celery.worker.consumer:INFO: > Connected to qpid://mysrv:5672// > <<a href="http://srpadinf0244.mgt.insim.biz:5672//" rel="noreferrer" target="_blank">http://srpadinf0244.mgt.insim.biz:5672//</a>>____ > > May 04 05:11:07 mysrv pulp[4776]: kombu.transport.qpid:INFO: > Connected to qpid with SASL mechanism ANONYMOUS____ > > Hint: Some lines were ellipsized, use -l to show in full.____ > > __ __ > > __ __ > > # systemctl status pulp_worker-3____ > > ? pulp_worker-3.service - Pulp Worker #3____ > > Loaded: loaded (/run/systemd/system/pulp_worker-3.service; > static; vendor preset: disabled)____ > > Active: active (running) since Wed 2016-05-04 05:11:21 CEST; 59s > ago____ > > Main PID: 4798 (celery)____ > > CGroup: /system.slice/pulp_worker-3.service____ > > +-4798 /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-3@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var...____ > > +-4811 /usr/bin/python /usr/bin/celery worker -n > reserved_resource_worker-3@%h -A pulp.server.async.app -c 1 --events > --umask 18 --pidfile=/var...____ > > __ __ > > May 04 05:11:22 mysrv celery[4798]: - ** ---------- .> transport: > qpid://mysrv:5672// <<a href="http://srpadinf0244.mgt.insim.biz:5672//" rel="noreferrer" target="_blank">http://srpadinf0244.mgt.insim.biz:5672//</a>>____ > > May 04 05:11:22 mysrv celery[4798]: - ** ---------- .> results: > disabled____ > > May 04 05:11:22 mysrv celery[4798]: - *** --- * --- .> concurrency: > 1 (prefork)____ > > May 04 05:11:22 mysrv celery[4798]: -- ******* ----____ > > May 04 05:11:22 mysrv celery[4798]: --- ***** ----- [queues]____ > > May 04 05:11:22 mysrv celery[4798]: -------------- .> > celery exchange=celery(direct) key=celery____ > > May 04 05:11:22 mysrv celery[4798]: .> > reserved_resource_worker-3@srpadinf0244.mgt.insim.biz.dq > exchange=C.dq(direct) key=rese...s <<a href="http://sim.biz/" rel="noreferrer" target="_blank">http://sim.biz/</a>>rv____ > > May 04 05:11:22 mysrv pulp[4798]: kombu.transport.qpid:INFO: > Connected to qpid with SASL mechanism ANONYMOUS____ > > May 04 05:11:22 mysrv pulp[4798]: celery.worker.consumer:INFO: > Connected to qpid://mysrv:5672// > <<a href="http://srpadinf0244.mgt.insim.biz:5672//" rel="noreferrer" target="_blank">http://srpadinf0244.mgt.insim.biz:5672//</a>>____ > > May 04 05:11:22 mysrv pulp[4798]: kombu.transport.qpid:INFO: > Connected to qpid with SASL mechanism ANONYMOUS____ > > Hint: Some lines were ellipsized, use -l to show in full.____ > > __ __ > > *Now I have no tasks queued at all. I ran the repo sync again. and > again it went to waiting.....____* > > __ __ > > # pulp-admin tasks list____ > > +----------------------------------------------------------------------+____ > > Tasks____ > > +----------------------------------------------------------------------+____ > > __ __ > > No tasks found____ > > __ __ > > # pulp-admin rpm repo sync run > --repo-id=rhel-6-server-supplementary-rpms____ > > +----------------------------------------------------------------------+____ > > Synchronizing Repository [rhel-6-server-supplementary-rpms]____ > > +----------------------------------------------------------------------+____ > > __ __ > > This command may be exited via ctrl+c without affecting the request.____ > > __ __ > > __ __ > > [/]____ > > Waiting to begin... > > > > *//* > > > On Wed, May 4, 2016 at 1:52 AM, Brian Bouterse <<a href="mailto:bbouters@redhat.com">bbouters@redhat.com</a> <div><div class="h5">> <mailto:<a href="mailto:bbouters@redhat.com">bbouters@redhat.com</a>>> wrote: > > Kodiak is right that the second task stuck at "Waiting to Begin" is > likely waiting behind another operation on that same repo. > Canceling the > one prior will likely allow the later one to start. > > How many workers are running and how many do you expect? You can see > what Pulp thinks with: `pulp-admin status` > > You can compare that to your pulp processes on all of your Pulp > servers > with `sudo ps -awfux | grep celery`. > > Also you can look at the task details with -vv to see the worker the > halted task is assigned to. Something like `pulp-admin -vv tasks > details > --task-id 03842c9d-e053-4a6f-a4c4-2d7302be9c8c.` > > Unfortunately you'll have to see the worker in the raw response > with -vv > because of [0]. > > [0]: <a href="https://pulp.plan.io/issues/1832" rel="noreferrer" target="_blank">https://pulp.plan.io/issues/1832</a> > > -Brian > > > On 05/03/2016 11:53 AM, Kodiak Firesmith wrote: > > I believe you may need to cancel the pending repo sync task before you > > can delete the repo. Maybe try: > > pulp-admin tasks cancel --task-id=2d776d63-fd8a-4e0a-8f32-d2276c85187c > > pulp-admin tasks cancel --task-id=03842c9d-e053-4a6f-a4c4-2d7302be9c8c > > > > Then: > > pulp-admin rpm repo delete --repo-id=rhel-6-server-supplementary-rpms > > > > > > On Tue, May 3, 2016 at 11:47 AM, Mallick, Samiron > > <<a href="mailto:samiron.mallick@gmail.com">samiron.mallick@gmail.com</a> <mailto:<a href="mailto:samiron.mallick@gmail.com">samiron.mallick@gmail.com</a>> </div></div>> <mailto:<a href="mailto:samiron.mallick@gmail.com">samiron.mallick@gmail.com</a> <div><div class="h5">> <mailto:<a href="mailto:samiron.mallick@gmail.com">samiron.mallick@gmail.com</a>>>> wrote: > > > > Could anyone please tell me what went wrong with the > repository. One > > of my EL7 server registered and was able to fetch contents > from CDN. > > Recently I found one of the repo stuck after downloading > RPMs. It’s > > never ending. I rebooted my server, cancelled tasks, > deleted the > > repo and recreated, but no luck. No if I run sync, it > directly going > > to waiting stage. Earlier I observed it was starting the > task but > > the start time was same as I ran the job first time. Even > now I am > > not able to delete the repo as well as it is showing > “Waiting to > > begin”. I am running Pulp v2.8. Any idea would be greatly > appreciated. > > > > > > > > # rpm -qa pulp-server > > > > pulp-server-2.8.2-1.el7.noarch > > > > > > > > # pulp-admin rpm repo sync run > > --repo-id=rhel-6-server-supplementary-rpms > > > > > +----------------------------------------------------------------------+ > > > > Synchronizing Repository > [rhel-6-server-supplementary-rpms] > > > > > +----------------------------------------------------------------------+ > > > > > > > > This command may be exited via ctrl+c without affecting > the request. > > > > > > > > > > > > Downloading metadata... > > > > [\] > > > > ... completed > > > > > > > > Downloading repository content... > > > > [-] > > > > [==================================================] 100% > > > > RPMs: 0/0 items > > > > Delta RPMs: 0/0 items > > > > > > > > ... completed > > > > > > > > Downloading distribution files... > > > > [==================================================] 100% > > > > Distributions: 0/0 items > > > > ... completed > > > > > > > > Importing errata... > > > > [/] > > > > ... completed > > > > > > > > Importing package groups/categories... > > > > [-] > > > > ... completed > > > > > > > > Cleaning duplicate packages... > > > > [|] > > > > ... completed > > > > > > > > *** AND STUCK HERE *** > > > > > > > > # pulp-admin tasks list > > > > > +----------------------------------------------------------------------+ > > > > Tasks > > > > > +----------------------------------------------------------------------+ > > > > > > > > Operations: sync > > > > Resources: rhel-6-server-supplementary-rpms (repository) > > > > State: Running > > > > Start Time: 2016-05-03T07:06:36Z > > > > Finish Time: Incomplete > > > > Task Id: 2d776d63-fd8a-4e0a-8f32-d2276c85187c > > > > > > > > Operations: publish > > > > Resources: rhel-6-server-supplementary-rpms (repository) > > > > State: Waiting > > > > Start Time: Unstarted > > > > Finish Time: Incomplete > > > > Task Id: 03842c9d-e053-4a6f-a4c4-2d7302be9c8c > > > > > > > > # date > > > > Tue May 3 09:22:30 CEST 2016 > > > > # pulp-admin rpm repo sync schedules list > > --repo-id=rhel-6-server-supplementary-rpms > > > > > +----------------------------------------------------------------------+ > > > > Schedules > > > > > +----------------------------------------------------------------------+ > > > > > > > > There are no schedules defined for this operation. > > > > > > > > # pulp-admin rpm repo delete > --repo-id=rhel-6-server-supplementary-rpms > > > > This command may be exited via ctrl+c without affecting > the request. > > > > > > > > > > > > [-] > > > > Running... > > > > [-] > > > > Waiting to begin... > > > > > > > > *** AND STUCK HERE *** > > > > > > _______________________________________________ > > Pulp-list mailing list > > <a href="mailto:Pulp-list@redhat.com">Pulp-list@redhat.com</a> <mailto:<a href="mailto:Pulp-list@redhat.com">Pulp-list@redhat.com</a>> </div></div>> <mailto:<a href="mailto:Pulp-list@redhat.com">Pulp-list@redhat.com</a> <mailto:<a href="mailto:Pulp-list@redhat.com">Pulp-list@redhat.com</a>>> > > <a href="https://www.redhat.com/mailman/listinfo/pulp-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a> > > > > > > > > > > _______________________________________________ > > Pulp-list mailing list > > <a href="mailto:Pulp-list@redhat.com">Pulp-list@redhat.com</a> <mailto:<a href="mailto:Pulp-list@redhat.com">Pulp-list@redhat.com</a>> > > <a href="https://www.redhat.com/mailman/listinfo/pulp-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a> > > > > _______________________________________________ > Pulp-list mailing list > <a href="mailto:Pulp-list@redhat.com">Pulp-list@redhat.com</a> <mailto:<a href="mailto:Pulp-list@redhat.com">Pulp-list@redhat.com</a>> > <a href="https://www.redhat.com/mailman/listinfo/pulp-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a> > > > </blockquote></div> </div></div>