[Spacewalk-list] spacewalk-proxy performance with large repodata files

[Andy Warring]

Hi,

We have a Spacwalk v2.5 instance in Europe which serves a number of
Spacewalk Proxies in the US, EMEA and ASIA.

Both the Master and Proxies are running CentOS 6.10

I have noticed issues when a client requests a large file from the proxy
and we see an error on the proxy like this:

/var/log/httpd/error_log

error_log:[Thu Sep 13 14:23:22 2018] [error] [client 10.122.20.181]
mod_wsgi (pid=29734): Exception occurred processing WSGI script
'/usr/share/rhn/wsgi/xmlrpc.py'.
error_log:[Thu Sep 13 14:23:22 2018] [error] [client 10.122.20.181]
IOError: failed to write data

I believe the file in question was the other.xml.gz repodata file for our
CentOS 6 Updates channel which is 134mb in size.

When the client retries, it usually then works (but not always) but I would
like to implement auto-scaling instances where I need the Proxy to deliver
packages/repodata every time.

I also see this to a lesser extent with rpm packages, so I have been
reading about the Proxy-precache method (now this addresses rpm's but does
it also cover the repo metadata?).

If this is just the RPM files, what options do I have regarding the
metadata? (setting up another rsync job, or wget the file from the master
<- this is something I've done in the past).

I have looked at the timeouts for httpd and cache parameters for squid (
using https://svelab.com/2018/01/28/deploying-spacewalk-proxy/ as a
reference) but that hasn't appeared to help with this situation.

I am also looking at replacing our current AWS c4.large / magnetic disks
with c5.large, SSD, ENA network drivers.

Any help/advice would be appreciated, certainly with the repodata question.

many thanks

Andy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/spacewalk-list/attachments/20180917/35529501/attachment.htm>