[Spacewalk-list] Spacewalk - PostgreSQL High I/O

[Matt Moldvan]

Well for a smaller number of clients (<500-1,000) separating out the
database might be more work than it's worth (though good experience).
First step I would say is to take a look at increasing the logging in
Postgres to get an idea where the slowness is occurring.  Also, something
like atop or another monitoring solution can give you an idea of when the
spikes occur, maybe allowing you to correlate other system or maintenance
activities.

On Tue, Oct 11, 2016 at 8:37 AM Allan Moraes <allan at allanmoraes.com.br>
wrote:

> Thank you for the tips,
>
> In this case there is available 6GB of memory and the high I/O occur at
> the postgres disk. Other disks, the I/O is normal. The system not is using
> swap and there is 3GB of swap.
>
> I will separate the postgre and apply your tips.
>
> 2016-10-10 21:58 GMT-03:00 Matt Moldvan <matt at moldvan.com>:
>
> We have about 6,000 systems to manage and it was unusable otherwise... I
> had way too much trouble trying to get OSAD to work through proxies and F5
> load balancers, so I had to end up pointing them all to two masters that
> are still using the same Postgres database VM.  I was also toying with
> having the database be the back end for OSAD, so with that in mind the
> number of concurrent clients would often reach higher than usual
> numbers...  I tried a lot of different things to get Spacewalk stable,
> usable, and have proper failover, so I don't know that any of my
> recommendations or environment specific settings might be a silver bullet
> for anyone else, but it can't hurt to try, and learn in the process.
>
> On Mon, Oct 10, 2016 at 6:23 PM Paul Robert Marino <prmarino1 at gmail.com>
> wrote:
>
> tuning for 5000 clients is nuts that would hurt your performance
> try running pgtune for about 50 to maybe 500 clients max, but I try
> the lower setting first.
> Now lets talk about the high IO that usually happens when you don't
> have enough working memory in PostgreSQL's configuration. When that
> happens PostgreSQL creates temp files that are slow and do a lot of
> write IO during read operations because it will have to swap the data
> out to the temp files, note setting the number of connections too high
> would exacerbate that issue f its the root cause.
> By the way I managed up to 400 with spacewalk and never had to disable
> the snapshots.
>
>
> On Mon, Oct 10, 2016 at 4:48 PM, Matt Moldvan <matt at moldvan.com> wrote:
> > I had similar issues and ended up first breaking out the database to it's
> > own VM, then increasing the Postgres debug logs.  I saw that there were a
> > large number of operations running against the snapshot tables, with
> locks
> > and so on being set for a long period of time.  In /etc/rhn/rhn.conf, try
> > disabling snapshots with:
> >
> > enable_snapshots = 0
> >
> > I also did quite a bit of Postgres tuning using pgtune, for 5,000
> clients or
> > so:
> > pgtune -i data/postgresql.conf  -o ./data/postgresql.conf.new -c 5000
> >
> > Another thing that may help is installing pgbadger to analyze your
> Postgres
> > logs... it has some nice visualizations of the types of queries and
> tables
> > involved, which may point you in the right direction if snapshots aren't
> the
> > reason for the high utilization.
> > https://github.com/dalibo/pgbadger
> >
> > Hope that helps.
> >
> > On Mon, Oct 10, 2016 at 4:06 PM Konstantin Raskoshnyi <
> konrasko at gmail.com>
> > wrote:
> >>
> >> Because all your systems request information from SP, and default
> >> installation doesn't make any sense if you have more that 50 machines,
> so
> >> you need to tyne postgres, tomcat & linux itself
> >>
> >> On Mon, Oct 10, 2016 at 12:34 PM, Allan Moraes <
> allan at allanmoraes.com.br>
> >> wrote:
> >>>
> >>> Hi
> >>> In my CentOS 7 server, is installed the spacewalk 2.4 and PostgreSQL
> from
> >>> default installation. Via iotop, my postgresql write a lot of
> informations,
> >>> during all day. Why this occur?
> >>>
> >>> _______________________________________________
> >>> Spacewalk-list mailing list
> >>> Spacewalk-list at redhat.com
> >>> https://www.redhat.com/mailman/listinfo/spacewalk-list
> >>
> >>
> >> _______________________________________________
> >> Spacewalk-list mailing list
> >> Spacewalk-list at redhat.com
> >> https://www.redhat.com/mailman/listinfo/spacewalk-list
> >
> >
> > _______________________________________________
> > Spacewalk-list mailing list
> > Spacewalk-list at redhat.com
> > https://www.redhat.com/mailman/listinfo/spacewalk-list
>
> _______________________________________________
> Spacewalk-list mailing list
> Spacewalk-list at redhat.com
> https://www.redhat.com/mailman/listinfo/spacewalk-list
>
>
> _______________________________________________
> Spacewalk-list mailing list
> Spacewalk-list at redhat.com
> https://www.redhat.com/mailman/listinfo/spacewalk-list
>
>
>
>
> --
>
> Atenciosamente...
>
> *Allan Moraes*
> *- Linux Consulting at Venda e Cia <http://vendaecia.com/>*
> *- Founder and Editor at MySQL Box <http://www.mysqlbox.com.br/>*
> *- Linux System Administrador and DBA MySQL at Umbler
> <https://www.umbler.com/>*
>
> *Cel: (51) 81885480*
> *E-mail: allan at mysqlbox.com.br <allan at mysqlbox.com.br>*
> *Skype: allan at allanmoraes.com.br <allan at allanmoraes.com.br>*
> _______________________________________________
> Spacewalk-list mailing list
> Spacewalk-list at redhat.com
> https://www.redhat.com/mailman/listinfo/spacewalk-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/spacewalk-list/attachments/20161011/95059ea3/attachment.htm>