<html><head><title></title></head><body><div class="rte-style-maintainer rte-pre-wrap" data-color="global-default" bbg-color="default" data-bb-font-size="medium"bbg-font-size="medium" style="white-space: pre-wrap; font-size: small; font-family: "Courier New", Courier, "BB.FixedWidth";"style="white-space: pre-wrap; font-size: small; font-family: "Courier New", Courier, "BB.FixedWidth"; color: rgb(0, 0, 0);">Thanks Mike. This works. Set the timeout to 120s. The query of all packages in rhel7 actually took less than a minute. Default 30s seems a little too short.<br><br><div class="rte-style-maintainer" data-color="global-default"bbg-color="default" data-bb-font-size="medium" bbg-font-size="medium" style="font-size: small; font-family: "Courier New", Courier, "BB.FixedWidth";"style="font-size: small; font-family: "Courier New", Courier, "BB.FixedWidth"; color: rgb(0, 0, 0);"><div><div class="bbg-rte-fold-content" data-header="From: mikedep333@redhat.com At: 02/04/20 13:32:26" data-digest="From: mikedep333@redhat.com At: 02/04/20 13:32:26" style=""><div class="bbg-rte-fold-summary">From: mikedep333@redhat.com At: 02/04/20 13:32:26</div>To: <a spellcheck="false" bbg-destination="mailto:dkliban@redhat.com" href="mailto:dkliban@redhat.com"> dkliban@redhat.com</a><br>Cc: <a spellcheck="false"bbg-destination="mailto:bli111@bloomberg.net" href="mailto:bli111@bloomberg.net"> Bin Li (BLOOMBERG/ 120 PARK ) </a>, <a spellcheck="false"bbg-destination="mailto:pulp-list@redhat.com" href="mailto:pulp-list@redhat.com"> pulp-list@redhat.com</a><br>Subject: Re: [Pulp-list] pulp 3 list packages</div><br></div><div class="rte-internet-block-wrapper" style="background: white; color: black; font-family: Arial, "BB.Proportional"; font-size: small; white-space: normal;"><div class="rte-internet-block"><blockquote><div dir="ltr"><div>Hi Bin,</div><div><br></div><div>Did this work for you?</div><div><br></div><div>if so, it sounds like we should look into always having ansible-pulp / the unit file pass a value to gunicorn (and make it configurable.)</div><div><br></div><div>-Mike<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Jan 31, 2020 at 10:27 AM Dennis Kliban <<a spellcheck="false"bbg-destination="mailto:rte:bind" class="" href="mailto:dkliban@redhat.com" data-destination="mailto:rte:bind">dkliban@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>This can be modified using the systemd unit file[0]. Gunicorn takes a -t argument. It defaults to 30 seconds. <br></div><div><br></div><div>[0] <a spellcheck="false" bbg-destination="rte:bind" class=""href="https://github.com/pulp/ansible-pulp/blob/master/roles/pulp/templates/pulpcore-api.service.j2#L13"data-destination="rte:bind">https://github.com/pulp/ansible-pulp/blob/master/roles/pulp/templates/pulpcore-api.service.j2#L13</a></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Jan 31, 2020 at 9:26 AM Bin Li (BLOOMBERG/ 120 PARK) <<a spellcheck="false"bbg-destination="mailto:rte:bind" class="" href="mailto:bli111@bloomberg.net" data-destination="mailto:rte:bind">bli111@bloomberg.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="font-size: small; font-family: "Courier New", Courier, "BB.FixedWidth"; white-space: pre-wrap;">Thanks Brian.<div><br></div><div>It looks like the worker timeout.</div><div><br></div><div>Jan 31 09:19:30 pulpp-ob-581 gunicorn[147883]: [2020-01-31 09:19:30 -0500] [147883] [CRITICAL] WORKER TIMEOUT (pid:147898)</div><div>Jan 31 09:19:30 pulpp-ob-581 gunicorn[147883]: [2020-01-31 14:19:30 +0000] [147898] [INFO] Worker exiting (pid: 147898)</div><div>Jan 31 09:19:31 pulpp-ob-581 gunicorn[147883]: [2020-01-31 09:19:31 -0500] [161199] [INFO] Booting worker with pid: 161199</div><div><br></div><div>How do I change the timeout setting in settings.py?</div><div><br><br><div style="font-size: small; font-family: "Courier New", Courier, "BB.FixedWidth";"><div><div><div>From: <a spellcheck="false" bbg-destination="mailto:rte:bind"class="" href="mailto:bmbouter@redhat.com" data-destination="mailto:rte:bind">bmbouter@redhat.com</a> At: 01/31/20 04:18:14</div>To: <a spellcheck="false" bbg-destination="mailto:bli111@bloomberg.net" class="" href="mailto:bli111@bloomberg.net" data-destination="mailto:bli111@bloomberg.net"> Bin Li (BLOOMBERG/ 120 PARK ) </a><br>Cc: <a spellcheck="false"bbg-destination="mailto:pulp-list@redhat.com" class="" href="mailto:pulp-list@redhat.com" data-destination="mailto:pulp-list@redhat.com"> pulp-list@redhat.com</a><br>Subject: Re: [Pulp-list] pulp 3 list packages</div><br></div><div style="background: none 0% 0% repeat scroll white; color: black; font-family: Arial, "BB.Proportional"; font-size: small; white-space: normal;"><div><blockquote><div dir="ltr"><div>What do the logs say about why the gunicorn process serving pulp-api is dying? Would you want to file an issue <a spellcheck="false"bbg-destination="rte:bind" class="" href="https://pulp.plan.io/issues/new" data-destination="rte:bind">https://pulp.plan.io/issues/new</a> so we can do some testing?</div><div><br></div><div>As an aside, I recommend using paging when pulling so many items from an API. You could decompose your large request to more, smaller requests like:</div><div><br></div><div>http GET localhost/pulp/api/v3/content/rpm/packages/ offset=0 limit==10000 repository_version==/pulp/api/v3/repositories/rpm/rpm/2f46d319-7997-4e86-b159-8babee4aba19/versions/1/ --timeout=200<br></div><div>http GET localhost/pulp/api/v3/content/rpm/packages/ offset=10000 limit==10000 repository_version==/pulp/api/v3/repositories/rpm/rpm/2f46d319-7997-4e86-b159-8babee4aba19/versions/1/ --timeout=200</div><div><br></div><div>What's interesting about more, smaller requests is you can likely get the data out of Pulp a lot faster since you can engage more gunicorn processes in parallel. Conceptually one large query is attractive though, so maybe we could improve that if you file it.</div><div><br></div><div>Another idea is to limit which fields are being returned to get at the data you need faster.<br></div><div><br></div><div>All the best,</div><div>Brian</div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jan 30, 2020 at 2:46 PM Bin Li (BLOOMBERG/ 120 PARK) <<a spellcheck="false"bbg-destination="mailto:rte:bind" class="" href="mailto:bli111@bloomberg.net" data-destination="mailto:rte:bind">bli111@bloomberg.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="font-size: small; font-family: "Courier New", Courier, "BB.FixedWidth"; white-space: pre-wrap;">The rhel 7 servers rpm repo has more than 26k packages. I got an "502 Bad Gateway" error if I tried to list all of them<div><br><div>http GET localhost/pulp/api/v3/content/rpm/packages/ limit==20000 repository_version==/pulp/api/v3/repositories/rpm/rpm/2f46d319-7997-4e86-b159-8babee4aba19/versions/1/ --timeout=200</div></div><div><br></div><div>What could cause this? Is there a fix?</div><div><br></div><div>Thanks</div></div>_______________________________________________<br>Pulp-list mailing list<br><a spellcheck="false"bbg-destination="mailto:rte:bind" class="" href="mailto:Pulp-list@redhat.com" data-destination="mailto:rte:bind">Pulp-list@redhat.com</a><br><a spellcheck="false"bbg-destination="rte:bind" class="" href="https://www.redhat.com/mailman/listinfo/pulp-list"data-destination="rte:bind">https://www.redhat.com/mailman/listinfo/pulp-list</a></blockquote></div></blockquote><br></div></div></div></div></div>_______________________________________________<br>Pulp-list mailing list<br><a spellcheck="false"bbg-destination="mailto:rte:bind" class="" href="mailto:Pulp-list@redhat.com" data-destination="mailto:rte:bind">Pulp-list@redhat.com</a><br><a spellcheck="false"bbg-destination="rte:bind" class="" href="https://www.redhat.com/mailman/listinfo/pulp-list"data-destination="rte:bind">https://www.redhat.com/mailman/listinfo/pulp-list</a></blockquote></div> _______________________________________________<br>Pulp-list mailing list<br><a spellcheck="false"bbg-destination="mailto:rte:bind" class="" href="mailto:Pulp-list@redhat.com" data-destination="mailto:rte:bind">Pulp-list@redhat.com</a><br><a spellcheck="false"bbg-destination="rte:bind" class="" href="https://www.redhat.com/mailman/listinfo/pulp-list"data-destination="rte:bind">https://www.redhat.com/mailman/listinfo/pulp-list</a></blockquote></div><br><br>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><p style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-weight:bold;margin:0px;padding:0px;font-size:14px;text-transform:capitalize"><span>Mike</span> <span>DePaulo</span></p><p style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-size:12px;margin:0px 0px 4px;text-transform:capitalize">He / Him / His</p><p style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-size:12px;margin:0px;text-transform:capitalize"><span>Service Reliability Engineer, Pulp</span></p><p style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;margin:0px 0px 4px;font-size:12px"><a style="margin: 0px;"spellcheck="false" bbg-destination="https://www.redhat.com/" class="" href="https://www.redhat.com/" data-destination="https://www.redhat.com/">Red Hat</a></p><div style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-size:medium;margin-bottom:4px"></div><p style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;margin:0px;font-size:12px"><span>IM: <span>mikedep333</span></span></p><p style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-size:12px;margin:0px"><span>GPG: 51745404</span></p><div style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-size:medium;margin-top:12px"><table border="0"><tbody><tr><td width="100px"><a spellcheck="false" bbg-destination="https://www.redhat.com/" class="" href="https://www.redhat.com/"data-destination="https://www.redhat.com/"><img src="https://marketing-outfit-prod-images.s3-us-west-2.amazonaws.com/f5445ae0c9ddafd5b2f1836854d7416a/Logo-RedHat-Email.png"style="width: 90px;" width="90"></a></td></tr></tbody></table></div></div></div></blockquote><br></div></div></div></div></body></html>