[Pulp-list] pulp 2.8 repos went into waiting state and never ends

[Brian Bouterse]

Kodiak is right that the second task stuck at "Waiting to Begin" is
likely waiting behind another operation on that same repo. Canceling the
one prior will likely allow the later one to start.

How many workers are running and how many do you expect? You can see
what Pulp thinks with:  `pulp-admin status`

You can compare that to your pulp processes on all of your Pulp servers
with `sudo ps -awfux | grep celery`.

Also you can look at the task details with -vv to see the worker the
halted task is assigned to. Something like `pulp-admin -vv tasks details
--task-id 03842c9d-e053-4a6f-a4c4-2d7302be9c8c.`

Unfortunately you'll have to see the worker in the raw response with -vv
because of [0].

[0]: https://pulp.plan.io/issues/1832

-Brian


On 05/03/2016 11:53 AM, Kodiak Firesmith wrote:
> I believe you may need to cancel the pending repo sync task before you
> can delete the repo.  Maybe try:
>  pulp-admin tasks cancel --task-id=2d776d63-fd8a-4e0a-8f32-d2276c85187c
>  pulp-admin tasks cancel --task-id=03842c9d-e053-4a6f-a4c4-2d7302be9c8c
> 
> Then:
> pulp-admin rpm repo delete --repo-id=rhel-6-server-supplementary-rpms
> 
> 
> On Tue, May 3, 2016 at 11:47 AM, Mallick, Samiron
> <samiron.mallick at gmail.com <mailto:samiron.mallick at gmail.com>> wrote:
> 
>     Could anyone please tell me what went wrong with the repository. One
>     of my EL7 server registered and was able to fetch contents from CDN.
>     Recently I found one of the repo stuck after downloading RPMs. It’s
>     never ending. I rebooted my server, cancelled tasks, deleted the
>     repo and recreated, but no luck. No if I run sync, it directly going
>     to waiting stage. Earlier I observed it was starting the task but
>     the start time was same as I ran the job first time. Even now I am
>     not able to delete the repo as well as it is showing “Waiting to
>     begin”. I am running Pulp v2.8. Any idea would be greatly appreciated.
> 
>      
> 
>     # rpm -qa pulp-server
> 
>     pulp-server-2.8.2-1.el7.noarch
> 
>      
> 
>     # pulp-admin rpm repo sync run
>     --repo-id=rhel-6-server-supplementary-rpms
> 
>     +----------------------------------------------------------------------+
> 
>           Synchronizing Repository [rhel-6-server-supplementary-rpms]
> 
>     +----------------------------------------------------------------------+
> 
>      
> 
>     This command may be exited via ctrl+c without affecting the request.
> 
>      
> 
>      
> 
>     Downloading metadata...
> 
>     [\]
> 
>     ... completed
> 
>      
> 
>     Downloading repository content...
> 
>     [-]
> 
>     [==================================================] 100%
> 
>     RPMs:       0/0 items
> 
>     Delta RPMs: 0/0 items
> 
>      
> 
>     ... completed
> 
>      
> 
>     Downloading distribution files...
> 
>     [==================================================] 100%
> 
>     Distributions: 0/0 items
> 
>     ... completed
> 
>      
> 
>     Importing errata...
> 
>     [/]
> 
>     ... completed
> 
>      
> 
>     Importing package groups/categories...
> 
>     [-]
> 
>     ... completed
> 
>      
> 
>     Cleaning duplicate packages...
> 
>     [|]
> 
>     ... completed
> 
>      
> 
>     *** AND STUCK HERE ***
> 
>      
> 
>     # pulp-admin tasks list
> 
>     +----------------------------------------------------------------------+
> 
>                                      Tasks
> 
>     +----------------------------------------------------------------------+
> 
>      
> 
>     Operations:  sync
> 
>     Resources:   rhel-6-server-supplementary-rpms (repository)
> 
>     State:       Running
> 
>     Start Time:  2016-05-03T07:06:36Z
> 
>     Finish Time: Incomplete
> 
>     Task Id:     2d776d63-fd8a-4e0a-8f32-d2276c85187c
> 
>      
> 
>     Operations:  publish
> 
>     Resources:   rhel-6-server-supplementary-rpms (repository)
> 
>     State:       Waiting
> 
>     Start Time:  Unstarted
> 
>     Finish Time: Incomplete
> 
>     Task Id:     03842c9d-e053-4a6f-a4c4-2d7302be9c8c
> 
>      
> 
>     # date
> 
>     Tue May  3 09:22:30 CEST 2016
> 
>     # pulp-admin rpm repo sync schedules list
>     --repo-id=rhel-6-server-supplementary-rpms
> 
>     +----------------------------------------------------------------------+
> 
>                                    Schedules
> 
>     +----------------------------------------------------------------------+
> 
>      
> 
>     There are no schedules defined for this operation.
> 
>      
> 
>     # pulp-admin rpm repo delete --repo-id=rhel-6-server-supplementary-rpms
> 
>     This command may be exited via ctrl+c without affecting the request.
> 
>      
> 
>      
> 
>     [-]
> 
>     Running...
> 
>     [-]
> 
>     Waiting to begin...
> 
>      
> 
>     *** AND STUCK HERE ***
> 
> 
>     _______________________________________________
>     Pulp-list mailing list
>     Pulp-list at redhat.com <mailto:Pulp-list at redhat.com>
>     https://www.redhat.com/mailman/listinfo/pulp-list
> 
> 
> 
> 
> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
>