[Pulp-list] pulp 3 list packages

[Dennis Kliban]

This can be modified using the systemd unit file[0]. Gunicorn takes a -t
argument. It defaults to 30 seconds.

[0]
https://github.com/pulp/ansible-pulp/blob/master/roles/pulp/templates/pulpcore-api.service.j2#L13


On Fri, Jan 31, 2020 at 9:26 AM Bin Li (BLOOMBERG/ 120 PARK) <
bli111 at bloomberg.net> wrote:

> Thanks Brian.
>
> It looks like the worker timeout.
>
> Jan 31 09:19:30 pulpp-ob-581 gunicorn[147883]: [2020-01-31 09:19:30 -0500]
> [147883] [CRITICAL] WORKER TIMEOUT (pid:147898)
> Jan 31 09:19:30 pulpp-ob-581 gunicorn[147883]: [2020-01-31 14:19:30 +0000]
> [147898] [INFO] Worker exiting (pid: 147898)
> Jan 31 09:19:31 pulpp-ob-581 gunicorn[147883]: [2020-01-31 09:19:31 -0500]
> [161199] [INFO] Booting worker with pid: 161199
>
> How do I change the timeout setting in settings.py?
>
>
> From: bmbouter at redhat.com At: 01/31/20 04:18:14
> To: Bin Li (BLOOMBERG/ 120 PARK ) <bli111 at bloomberg.net>
> Cc: pulp-list at redhat.com
> Subject: Re: [Pulp-list] pulp 3 list packages
>
> What do the logs say about why the gunicorn process serving pulp-api is
> dying? Would you want to file an issue https://pulp.plan.io/issues/new so
> we can do some testing?
>
> As an aside, I recommend using paging when pulling so many items from an
> API. You could decompose your large request to more, smaller requests like:
>
> http GET localhost/pulp/api/v3/content/rpm/packages/ offset=0 limit==10000
> repository_version==/pulp/api/v3/repositories/rpm/rpm/2f46d319-7997-4e86-b159-8babee4aba19/versions/1/
> --timeout=200
> http GET localhost/pulp/api/v3/content/rpm/packages/ offset=10000
> limit==10000
> repository_version==/pulp/api/v3/repositories/rpm/rpm/2f46d319-7997-4e86-b159-8babee4aba19/versions/1/
> --timeout=200
>
> What's interesting about more, smaller requests is you can likely get the
> data out of Pulp a lot faster since you can engage more gunicorn processes
> in parallel. Conceptually one large query is attractive though, so maybe we
> could improve that if you file it.
>
> Another idea is to limit which fields are being returned to get at the
> data you need faster.
>
> All the best,
> Brian
>
>
>
> On Thu, Jan 30, 2020 at 2:46 PM Bin Li (BLOOMBERG/ 120 PARK) <
> bli111 at bloomberg.net> wrote:
>
>> The rhel 7 servers rpm repo has more than 26k packages. I got an "502 Bad
>> Gateway" error if I tried to list all of them
>>
>> http GET localhost/pulp/api/v3/content/rpm/packages/ limit==20000
>> repository_version==/pulp/api/v3/repositories/rpm/rpm/2f46d319-7997-4e86-b159-8babee4aba19/versions/1/
>> --timeout=200
>>
>> What could cause this? Is there a fix?
>>
>> Thanks
>> _______________________________________________
>> Pulp-list mailing list
>> Pulp-list at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-list
>
>
> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20200131/951e5fa1/attachment.htm>