<html><head><title></title></head><body><div class="rte-style-maintainer rte-pre-wrap" data-color="global-default" bbg-color="default" data-bb-font-size="medium"bbg-font-size="medium" style="white-space: pre-wrap; font-size: small; font-family: "Courier New", Courier, "BB.FixedWidth";"style="white-space: pre-wrap; font-size: small; font-family: "Courier New", Courier, "BB.FixedWidth"; color: rgb(0, 0, 0);">Thanks Brian.<div><br></div><div>It looks like the worker timeout.</div><div><br></div><div>Jan 31 09:19:30 pulpp-ob-581 gunicorn[147883]: [2020-01-31 09:19:30 -0500] [147883] [CRITICAL] WORKER TIMEOUT (pid:147898)</div><div>Jan 31 09:19:30 pulpp-ob-581 gunicorn[147883]: [2020-01-31 14:19:30 +0000] [147898] [INFO] Worker exiting (pid: 147898)</div><div>Jan 31 09:19:31 pulpp-ob-581 gunicorn[147883]: [2020-01-31 09:19:31 -0500] [161199] [INFO] Booting worker with pid: 161199</div><div><br></div><div>How do I change the timeout setting in settings.py?</div><div><br><br><div class="rte-style-maintainer" data-color="global-default"bbg-color="default" data-bb-font-size="medium" bbg-font-size="medium" style="font-size: small; font-family: "Courier New", Courier, "BB.FixedWidth";"style="font-size: small; font-family: "Courier New", Courier, "BB.FixedWidth"; color: rgb(0, 0, 0);"><div><div class="bbg-rte-fold-content" data-header="From: bmbouter@redhat.com At: 01/31/20 04:18:14" data-digest="From: bmbouter@redhat.com At: 01/31/20 04:18:14" style=""><div class="bbg-rte-fold-summary">From: bmbouter@redhat.com At: 01/31/20 04:18:14</div>To: <a spellcheck="false" bbg-destination="mailto:bli111@bloomberg.net" href="mailto:bli111@bloomberg.net"> Bin Li (BLOOMBERG/ 120 PARK ) </a><br>Cc: <a spellcheck="false"bbg-destination="mailto:pulp-list@redhat.com" href="mailto:pulp-list@redhat.com"> pulp-list@redhat.com</a><br>Subject: Re: [Pulp-list] pulp 3 list packages</div><br></div><div class="rte-internet-block-wrapper" style="background: white; color: black; font-family: Arial, "BB.Proportional"; font-size: small; white-space: normal;"><div class="rte-internet-block"><blockquote><div dir="ltr"><div>What do the logs say about why the gunicorn process serving pulp-api is dying? Would you want to file an issue <a spellcheck="false"bbg-destination="rte:bind" class="" href="https://pulp.plan.io/issues/new" data-destination="rte:bind">https://pulp.plan.io/issues/new</a> so we can do some testing?</div><div><br></div><div>As an aside, I recommend using paging when pulling so many items from an API. You could decompose your large request to more, smaller requests like:</div><div><br></div><div>http GET localhost/pulp/api/v3/content/rpm/packages/ offset=0 limit==10000 repository_version==/pulp/api/v3/repositories/rpm/rpm/2f46d319-7997-4e86-b159-8babee4aba19/versions/1/ --timeout=200<br></div><div>http GET localhost/pulp/api/v3/content/rpm/packages/ offset=10000 limit==10000 repository_version==/pulp/api/v3/repositories/rpm/rpm/2f46d319-7997-4e86-b159-8babee4aba19/versions/1/ --timeout=200</div><div><br></div><div>What's interesting about more, smaller requests is you can likely get the data out of Pulp a lot faster since you can engage more gunicorn processes in parallel. Conceptually one large query is attractive though, so maybe we could improve that if you file it.</div><div><br></div><div>Another idea is to limit which fields are being returned to get at the data you need faster.<br></div><div><br></div><div>All the best,</div><div>Brian</div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jan 30, 2020 at 2:46 PM Bin Li (BLOOMBERG/ 120 PARK) <<a spellcheck="false"bbg-destination="mailto:rte:bind" class="" href="mailto:bli111@bloomberg.net" data-destination="mailto:rte:bind">bli111@bloomberg.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="font-size: small; font-family: "Courier New", Courier, "BB.FixedWidth"; white-space: pre-wrap;">The rhel 7 servers rpm repo has more than 26k packages. I got an "502 Bad Gateway" error if I tried to list all of them<div><br><div>http GET localhost/pulp/api/v3/content/rpm/packages/ limit==20000 repository_version==/pulp/api/v3/repositories/rpm/rpm/2f46d319-7997-4e86-b159-8babee4aba19/versions/1/ --timeout=200</div></div><div><br></div><div>What could cause this? Is there a fix?</div><div><br></div><div>Thanks</div></div>_______________________________________________<br>Pulp-list mailing list<br><a spellcheck="false"bbg-destination="mailto:rte:bind" class="" href="mailto:Pulp-list@redhat.com" data-destination="mailto:rte:bind">Pulp-list@redhat.com</a><br><a spellcheck="false"bbg-destination="rte:bind" class="" href="https://www.redhat.com/mailman/listinfo/pulp-list"data-destination="rte:bind">https://www.redhat.com/mailman/listinfo/pulp-list</a></blockquote></div></blockquote><br></div></div></div></div></div></body></html>