[Pulp-list] Memory Usage

Brian Bouterse bbouters at redhat.com
Tue Apr 7 15:59:51 UTC 2015


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I generally think Mongo is going to be using significantly more memory
than pulp processes.

IIRC, the celery log files you referenced only log for a brief moment
in time before all logs redirect to syslog. See [0] for Pulp's logging
docs.

FYI there are two memory performance related bugs. 676 [1] has a fix
available in the 2.6.1 beta so you could see if that helps your memory
performance. Also 372 [2] is reported, but it has not been fixed.

[0]:
http://pulp.readthedocs.org/en/latest/user-guide/troubleshooting.html?highlight=logging#logging
[1]: https://pulp.plan.io/issues/676
[2]: https://pulp.plan.io/issues/372

- -Brian


On 04/07/2015 10:32 AM, Roth, Sandro wrote:
> Thanks for the quick reply!
> 
> Looks like its pulp's celery workers..
> 
> $ ps aux --sort -rss | head USER       PID %CPU %MEM    VSZ   RSS
> TTY      STAT START   TIME COMMAND apache    2450  0.0 46.4 6849480
> 5668064 ?     Sl   Mar25   6:59 /usr/bin/python -m celery.__main__
> worker -c 1 -n reserved_resource_worker-1 at server.local --events
> --app=pulp.server.async.app --loglevel=INFO
> --logfile=/var/log/pulp/reserved_resource_worker-1.log
> --pidfile=/var/run/pulp/reserved_resource_worker-1.pid apache
> 2493  0.0 19.0 6910896 2324800 ?     Sl   Mar25  11:39
> /usr/bin/python -m celery.__main__ worker -c 1 -n
> reserved_resource_worker-2 at server.local --events
> --app=pulp.server.async.app --loglevel=INFO
> --logfile=/var/log/pulp/reserved_resource_worker-2.log
> --pidfile=/var/run/pulp/reserved_resource_worker-2.pid apache
> 2503  0.0 10.7 2243588 1314404 ?     Sl   Mar25   3:05
> /usr/bin/python -m celery.__main__ worker -c 1 -n
> reserved_resource_worker-3 at server.local --events
> --app=pulp.server.async.app --loglevel=INFO
> --logfile=/var/log/pulp/reserved_resource_worker-3.log
> --pidfile=/var/run/pulp/reserved_resource_worker-3.pid apache
> 2419  0.0  4.8 1523648 595896 ?      Sl   Mar25   1:09
> /usr/bin/python -m celery.__main__ worker -c 1 -n
> reserved_resource_worker-0 at server.local --events
> --app=pulp.server.async.app --loglevel=INFO
> --logfile=/var/log/pulp/reserved_resource_worker-0.log
> --pidfile=/var/run/pulp/reserved_resource_worker-0.pid mongodb
> 2160  0.4  2.3 21494828 290668 ?     Sl   Mar25  80:58
> /usr/bin/mongod --quiet -f /etc/mongodb.conf run apache    4027
> 0.0  0.4 1091496 49016 ?       Sl   Apr05   1:34 (wsgi:pulp) apache
> 4025  0.0  0.3 1091496 47696 ?       Sl   Apr05   1:32 (wsgi:pulp) 
> apache    4026  0.0  0.3 1091496 47676 ?       Sl   Apr05   1:33
> (wsgi:pulp) root      2836  0.0  0.3 134528 40356 ?        Ss
> Mar25   0:45 /usr/bin/ruby /usr/bin/puppet agent
> 
> Any logs or stats I should collect before restarting the service?
> 
> These celery log files are all empty. And pulp related non-info
> logs in messages file are all because of download errors
> (nectar.downloaders.threaded).
> 
> 
> Sandro
> 
> -----Original Message----- From: pulp-list-bounces at redhat.com
> [mailto:pulp-list-bounces at redhat.com] On Behalf Of Randy Barlow 
> Sent: Dienstag, 7. April 2015 16:12 To: pulp-list at redhat.com 
> Subject: Re: [Pulp-list] Memory Usage
> 
> Hello Sandro,
> 
> MongoDB might be using the extra RAM. MongoDB uses memory mapped
> files[0] for its indices and collections. As a result, it will
> generally use all the RAM it can get. I suggest deploying MongoDB
> on a separate system (or VM) from the Pulp web server and workers
> to keep its RAM usage in a range that you prefer.
> 
> [0] https://en.wikipedia.org/wiki/Memory-mapped_file
> 
> 
> _______________________________________________ Pulp-list mailing
> list Pulp-list at redhat.com 
> https://www.redhat.com/mailman/listinfo/pulp-list
> 
> This email message and any attachments are confidential and may be
> privileged. If you are not the intended recipient, please notify us
> immediately and destroy the original transmittal. You are hereby
> notified that any review, copying or distribution of it is strictly
> prohibited. Thank you for your cooperation. Header information
> contained in E-mails to and from the company are monitored for
> operational reasons in accordance with the Swiss data protection
> act.
> 
> _______________________________________________ Pulp-list mailing
> list Pulp-list at redhat.com 
> https://www.redhat.com/mailman/listinfo/pulp-list
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJVI/73AAoJEK48cdELyEfyUyMH/0fi/Cxqq1ObRdinA09dnFsE
kWE+Q0LToNA9T1yxPSlsvVQnsqgc5LIJ3L3QdNcupvCgEiRMSv7TiTBIAxPMZuts
v3GHCfHtDQh9nT+xsVNjSLXZx95isuD5J0sxfeVmvHfjnYmo+MfKU32vzNc49twV
5ofUgGSADvgU0Mh2u5eQ/ZgtxqTAPnzN+tLT4jfpqHUrccSXSrZzhlrmTmFmVtC8
+zThshqdGeLh+7EhxkR4ohx7eEqUBZSAVjNmOh/Uz3kl4YwBvxTXVzeWQrgg2C7M
a3Sps8gr8PSMynAzq3lvMc13Kl5NOIw+/2FM1ORgeVGhDJ0ki7kIb68Saj9kihE=
=OxnC
-----END PGP SIGNATURE-----




More information about the Pulp-list mailing list