<div dir="ltr"><div>Hi Adam,</div><div><br></div><div>Thanks for sharing all of that and for opening that issue. Would you be willing to do one or two more things to help move it forward? Could you post the diff of the Apache config changes you made on the issue? Also if you'd be willing to open a PR I think it would go against this file. <a href="https://github.com/pulp/pulp_installer/blob/master/roles/pulp_webserver/templates/pulp-vhost.conf.j2">https://github.com/pulp/pulp_installer/blob/master/roles/pulp_webserver/templates/pulp-vhost.conf.j2</a> Can you link us to either of these you're able to do?<br></div><div><br></div><div>Regarding improving the content app's performance, @dalley has done some investigation there and we hope to build on that work to make some improvements.</div><div><br></div><div>Cheers,</div><div>Brian</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Feb 5, 2021 at 12:47 PM Winberg Adam <<a href="mailto:Adam.Winberg@smhi.se">Adam.Winberg@smhi.se</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir="ltr"> <div id="gmail-m_-6257280700189759964divtagdefaultwrapper" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif" dir="ltr"> <p>I raised a similar question here about pulp-content performance a short while ago, and also created <a href="https://pulp.plan.io/issues/8180" id="gmail-m_-6257280700189759964LPlnk875349" target="_blank">https://pulp.plan.io/issues/8180</a> since there may be a lack of documentation about this. </p> <p><br> </p> <p>In short, I've made a number of adjustments and among them was setting the '<span>disablereuse=on' setting in my proxypass declaration. I have not seen any performance issue, probably because the proxypass target is on localhost and not a remote machine. </span></p> <p><br> </p> <p>//Adam<span style="font-size:12pt"><span></span></span></p> <br> <br> <div style="color:rgb(0,0,0)"> <hr style="display:inline-block;width:98%"> <div id="gmail-m_-6257280700189759964divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> <a href="mailto:pulp-list-bounces@redhat.com" target="_blank">pulp-list-bounces@redhat.com</a> <<a href="mailto:pulp-list-bounces@redhat.com" target="_blank">pulp-list-bounces@redhat.com</a>> on behalf of Eric Helms <<a href="mailto:ehelms@redhat.com" target="_blank">ehelms@redhat.com</a>><br> <b>Sent:</b> 05 February 2021 17:29<br> <b>To:</b> pulp-list<br> <b>Subject:</b> [Pulp-list] Apache 502 proxy errors when performing Pulp 3 to Pulp 3 syncs</font> <div> </div> </div> <div> <div dir="ltr">Howdy, <div><br> </div> <div>Some quick background, over in the Katello project we deploy the pulpcore-content service via a unix socket with Apache serving as a reverse proxy. Today, we deploy pulpcore-content with two gunicorn workers. Tomorrow we are considering changing this to 2 * CPU + 1 per gunicorn documentation.</div> <div><br> </div> <div>The issue we are running into is intermittent 502s from Apache caused by being unable to make a connection to the underlying pulpcore-content app. This manifests itself primarily during a Pulp 3 to Pulp 3 sync. That is, when we sync our content proxy from the main servers Pulp 3. The sync can result in a large number of parallel connections back to the pulpcore-content application running on the main server.</div> <div><br> </div> <div>In an issue for aiohttp which is used by the project, and whose worker is used for gunicorn [1] they talk about the issues with Apache and aiohttp. In that issue there are two suggestions that I could extract:</div> <div><br> </div> <div> 1) set disablereuse=on the Apache reverse proxy declarations for the content app</div> <div> 2) change the default Apache worker type to be more like Nginx</div> <div><br> </div> <div>There are performance tradeoffs with #1, however, I do not fully grasp if they are relative to our primary use case when it comes to the content app.</div> <div><br> </div> <div>So, I am coming to the experts here to try to get some insight into what changes we should pursue to ensure the optimal default performance for our deployment. And to, as best as we can, limit these kind of intermittent failures to extreme cases. Because today, we see this intermittent failure with Pulp 3 running the same test suite we did with Pulp 2.</div> <div><br> </div> <div>Related, is there retry support built into syncing?</div> <div><br> </div> <div>Thanks!</div> <div><br> </div> <div>[1] <a href="https://github.com/aio-libs/aiohttp/issues/2687" id="gmail-m_-6257280700189759964LPlnk175241" target="_blank">https://github.com/aio-libs/aiohttp/issues/2687</a><br clear="all"> <div><br> </div> -- <br> <div dir="ltr"> <div dir="ltr"> <div> <div dir="ltr">Eric Helms <div>Principal Software Engineer</div> <div>Satellite</div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> _______________________________________________<br> Pulp-list mailing list<br> <a href="mailto:Pulp-list@redhat.com" target="_blank">Pulp-list@redhat.com</a><br> <a href="https://www.redhat.com/mailman/listinfo/pulp-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a></blockquote></div>