[Pulp-list] Content server Performance

[Brian Bouterse]

On Tue, Jun 22, 2021 at 11:56 AM Danny Sauer <danny.sauer at konghq.com> wrote:

> You can certainly run multiple instances of the content server.  It just
> needs a connection to the database and access to the storage.
>
Agreed, you could deploy additional content servers and have your
nginx/apache load balance them.


> Have you tuned the number of worker processes in Gunicorn?  It defaults to
> 1, but should almost certainly be increased for any sort of volume.
> https://docs.gunicorn.org/en/stable/settings.html#worker-processes
>
Pulp changed the default gunicorn worker processes to 8 maybe a release or
two ago. See the `pulp_content_workers` variable in the installer here
https://pulp-installer.readthedocs.io/en/latest/roles/pulp_content/#role-variables

>
> There are several moving pieces, but that's really all I had to touch here.
>
> --Danny
>
With pulpcore==3.14 there is a significant performance improvement being
reviewed now  https://pulp.plan.io/issues/8805  . In addition to resolving
it with methods like ^, when 3.14 comes out (scheduled for June 29th) it
would be great if you could report on if the improvements helped you.

>
> On Tue, Jun 22, 2021 at 10:34 AM Bin Li (BLOOMBERG/ 120 PARK) <
> bli111 at bloomberg.net> wrote:
>
>> We recently add more clients to use the pulp content server. The
>> processes run out the file descriptor first. We then increased both nginx
>> and pulp-content by creating a override.conf
>> /etc/systemd/system/pulpcore-content.service.d # cat override.conf
>> [Service]
>> LimitNOFILE=65536
>>
>> and updated nginx.conf
>> # Gunicorn docs suggest this value.
>> worker_processes 1;
>> events {
>> worker_connections 10000; # increase if you have lots of clients
>> accept_mutex off; # set to 'on' if nginx worker_processes > 1
>> }
>>
>> worker_rlimit_nofile 20000;
>>
>>
>> Now we are keep getting this error.
>> 2021/06/22 11:26:36 [error] 78373#0: *112823 upstream timed out (110:
>> Connection timed out) while connecting to upstream, client:
>>
>> It looks like pulp-content server cannot keep up with requests. Is there
>> anything we could do to increase the performance of the content server?
>> _______________________________________________
>> Pulp-list mailing list
>> Pulp-list at redhat.com
>> https://listman.redhat.com/mailman/listinfo/pulp-list
>
> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20210622/896039bb/attachment.htm>