[Spacewalk-list] Really slow unresponsive GUI

Fri Mar 15 07:35:44 UTC 2019

> Gesendet: Donnerstag, 14. März 2019 um 21:53 Uhr
> Von: "Lionel Caignec" <caignec at cines.fr>
> An: "Robert Paschedag" <robert.paschedag at web.de>
> Betreff: Re: [Spacewalk-list] Really slow unresponsive GUI
>
> no problem with cobbler is running like all other service.
> when i try to telnet to cobbler port i get process responding.
> 
> I'm really lost. Is it possible my kickstart profile are corrupted which give a bug with cobbler?
> 
> ----- Mail original -----
> De: "Robert Paschedag" <robert.paschedag at web.de>
> À: "spacewalk-list" <spacewalk-list at redhat.com>, "Lionel Caignec" <caignec at cines.fr>, "spacewalk-list" <spacewalk-list at redhat.com>
> Envoyé: Jeudi 14 Mars 2019 17:24:16
> Objet: Re: [Spacewalk-list] Really slow unresponsive GUI
> 
> Am 14. März 2019 12:58:23 MEZ schrieb Lionel Caignec <caignec at cines.fr>:
> > Hi,
> >
> >finally i managed to get the source of the bug, it's about then link
> >between tomcat and cobbler. The bug appears only when i try to view
> >kickstart profiles.
> >
> >In cobbler i get this error message : 
> >Wed Mar 13 12:06:01 2019 - INFO | Exception Info:
> >File "/usr/lib/python2.7/site-packages/cobbler/remote.py", line 1770,
> >in _dispatch
> >    return method_handle(*params)
> >File "/usr/lib/python2.7/site-packages/cobbler/remote.py", line 1598,
> >in token_check
> >    self.__validate_token(token)
> >File "/usr/lib/python2.7/site-packages/cobbler/remote.py", line 1510,
> >in __validate_token
> >    raise CX("invalid token: %s" % token)
> >
> >
> >In tomcat log i get this error :
> >mars 13, 2019 12:11:38 PM org.apache.catalina.core.StandardWrapperValve
> >invoke
> >GRAVE: Servlet.service() for servlet [action] in context with path
> >[/rhn] threw exception [java.lang.RuntimeException: XmlRpcException
> >calling cobbler.] with root cause

> >java.io.IOException: Server returned HTTP response code: 502 for URL:
> >http://FQDN:80/cobbler_api

Hmm....did you replace the original server hostname with "FQDN" here in the log output?

Robert

> >at
> >sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1894)
> >at
> >sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
> >  at redstone.xmlrpc.XmlRpcClient.handleResponse(XmlRpcClient.java:420)
> >    at redstone.xmlrpc.XmlRpcClient.endCall(XmlRpcClient.java:376)
> >    at redstone.xmlrpc.XmlRpcClient.invoke(XmlRpcClient.java:165)
> >at
> >com.redhat.rhn.manager.kickstart.cobbler.CobblerXMLRPCHelper.invokeMethod(CobblerXMLRPCHelper.java:69)
> >at
> >com.redhat.rhn.manager.kickstart.cobbler.CobblerLoginCommand.login(CobblerLoginCommand.java:52)
> >at
> >com.redhat.rhn.frontend.integration.IntegrationService.authorize(IntegrationService.java:114)
> >at
> >com.redhat.rhn.frontend.integration.IntegrationService.getAuthToken(IntegrationService.java:74)
> >at
> >com.redhat.rhn.manager.kickstart.cobbler.CobblerXMLRPCHelper.getConnection(CobblerXMLRPCHelper.java:93)
> >at
> >com.redhat.rhn.manager.kickstart.cobbler.CobblerXMLRPCHelper.getConnection(CobblerXMLRPCHelper.java:83)
> >at
> >com.redhat.rhn.manager.kickstart.KickstartLister.listCobblerProfiles(KickstartLister.java:436)
> >at
> >com.redhat.rhn.frontend.action.kickstart.KickstartsSetupAction.execute(KickstartsSetupAction.java:64)
> >at
> >org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:425)
> >at
> >org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:229)
> >at
> >com.redhat.rhn.frontend.struts.RhnRequestProcessor.process(RhnRequestProcessor.java:105)
> >at
> >org.apache.struts.action.ActionServlet.process(ActionServlet.java:1926)
> >at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:451)
> >    at javax.servlet.http.HttpServlet.service(HttpServlet.java:624)
> >    at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
> >at
> >org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> >at
> >org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> >at
> >org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> >at
> >org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> >at
> >org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> >at
> >com.redhat.rhn.frontend.servlets.AuthFilter.doFilter(AuthFilter.java:107)
> >at
> >org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> >at
> >org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> >at
> >com.opensymphony.sitemesh.webapp.SiteMeshFilter.obtainContent(SiteMeshFilter.java:129)
> >at
> >com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:77)
> >at
> >org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> >at
> >org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> >at
> >com.redhat.rhn.frontend.servlets.LocalizedEnvironmentFilter.doFilter(LocalizedEnvironmentFilter.java:67)
> >at
> >org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> >at
> >org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> >at
> >com.redhat.rhn.frontend.servlets.EnvironmentFilter.doFilter(EnvironmentFilter.java:101)
> >at
> >org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> >at
> >org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> >at
> >com.redhat.rhn.frontend.servlets.SessionFilter.doFilter(SessionFilter.java:58)
> >at
> >org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> >at
> >org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> >at
> >com.redhat.rhn.frontend.servlets.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:97)
> >at
> >org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> >at
> >org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> >at
> >org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
> >at
> >org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
> >at
> >org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506)
> >at
> >org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
> >at
> >org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
> >at
> >org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
> >at
> >org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> >at
> >org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
> >   at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:190)
> >at
> >org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
> >at
> >org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
> >at
> >java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >at
> >java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >at
> >org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> >    at java.lang.Thread.run(Thread.java:748)
> >
> >
> >Someone have an idea please?
> >
> >Regards Lionel
> >
> >----- Mail original -----
> >De: "Lionel Caignec" <caignec at cines.fr>
> >À: "spacewalk-list" <spacewalk-list at redhat.com>
> >Envoyé: Lundi 11 Mars 2019 10:14:05
> >Objet: Re: [Spacewalk-list] Really slow unresponsive GUI
> >
> >Yes i even rebooted the server.
> >
> >Here is the exact error message in sync log : 
> >ERROR: (23, 'ERROR: duplicate key value violates unique constraint
> >"rhn_cnp_cid_nid_uq"', 'Could not update database entry.')
> >
> >
> >I double checked this morning all is good until i click on "kickstart
> >--> distributions" or "kickstart --> profiles" then all server is
> >unresponsive.
> >
> >So my idea is to create a trying delete "kickstart profile" but i
> >cannot do it with gui/api so can i delete in database directly? In
> >which table ?
> >
> >
> >Regards,
> >Lionel.
> >
> >----- Mail original -----
> >De: "P Cookson" <P.Cookson at bham.ac.uk>
> >À: "spacewalk-list" <spacewalk-list at redhat.com>
> >Envoyé: Lundi 11 Mars 2019 09:22:57
> >Objet: Re: [Spacewalk-list] Really slow unresponsive GUI
> >
> >Just checking, did you re-start the Taskomatic service after changing
> >the setting too?
> >
> >Regards
> >Phil
> >
> >-----Original Message-----
> >From: spacewalk-list-bounces at redhat.com
> ><spacewalk-list-bounces at redhat.com> On Behalf Of caignec at cines.fr
> >Sent: 08 March 2019 15:56
> >To: spacewalk-list <spacewalk-list at redhat.com>
> >Subject: Re: [Spacewalk-list] Really slow unresponsive GUI
> >
> >Hi Phil,
> >
> >thank for your help. No error about java heap space, but tried to
> >configure taskomatic memory like you said without success.
> >
> >I setup the kickstart part long time ago, just using satellite redhat
> >doc.
> >
> >----- Mail original -----
> >De: "P Cookson" <P.Cookson at bham.ac.uk>
> >À: "spacewalk-list" <spacewalk-list at redhat.com>
> >Envoyé: Vendredi 8 Mars 2019 16:43:23
> >Objet: Re: [Spacewalk-list] Really slow unresponsive GUI
> >
> >Hi Lionel
> >
> >Have you looked in /var/log/rhn/rhn_taskomatic_daemon.log too? After
> >seeing "java.lang.OutOfMemoryError: Java heap space" messages, in
> >there, I changed config for Taskomatic but not sure if this is relevant
> >for you?
> >
> >Current maxmemory (Xmx) can be seen by checking status of Taskomatic:
> >
> ># systemctl status taskomatic.service
> >
> >I increased from 256 to 4096:
> >
> ># grep taskomatic.java.maxmemory /etc/rhn/rhn.conf
> >taskomatic.java.maxmemory=4096
> >#
> >
> >Then re-started Spacewalk:
> >
> ># spacewalk-service restart
> >
> >Also, I've noted you've said something about using Kickstart. After
> >patching for some time I need to start looking at provisioning, too, so
> >if you have any notes/links about setup that would be useful.
> >
> >
> >Regards
> >Phil
> >
> >-----Original Message-----
> >From: spacewalk-list-bounces at redhat.com
> ><spacewalk-list-bounces at redhat.com> On Behalf Of caignec at cines.fr
> >Sent: 08 March 2019 14:59
> >To: spacewalk-list <spacewalk-list at redhat.com>
> >Subject: [Spacewalk-list] Really slow unresponsive GUI
> >
> >Hi,
> >
> >i'm writing here because i've a strange behaviour with no clue on how
> >to resolve it. 
> >There is no load on CPU, ram is mostly free, no IO wait, no processus
> >stuck a 100%. In log file (tomcat/postresql,...) nothing about error or
> >warning.
> >No recent modification on the system.
> >
> >When i want to access spacewalk web interface (or API) this one
> >sometimes answer after a very long time, and sometimes get error 500.
> >Same behaviour with the spacecmd cli command.
> >
> >I really don't understand because i've another spacewalk on another
> >server and this one work like a charm.
> >
> >My setup : VM with 2vpcu 8Gb RAM near 200 clients spacewalk 2.8, used
> >only for patching and sometimes kickstart new server.
> > JAVA_OPTS -Xms2048m -Xmx2048m, tried with 4096 but no difference.
> >
> >I don't know which log to put here due to the fact i found no error
> >trace.
> >
> >For now the only solution I imagine is to backup and restore to a new
> >server freshly installed my spacewalk.
> >
> >Thank you for help.
> >
> >--
> >Lionel Caignec 
> >
> >Centre Informatique National de l' Enseignement Supérieur 
> >950 rue de Saint Priest 
> >34097 MONTPELLIER Cedex 5 
> >Tel : (33) 04 67 14 14 14
> >Fax : (33)04 67 52 37 63 
> >https://www.cines.fr
> >
> >_______________________________________________
> >Spacewalk-list mailing list
> >Spacewalk-list at redhat.com
> >https://www.redhat.com/mailman/listinfo/spacewalk-list
> >
> >_______________________________________________
> >Spacewalk-list mailing list
> >Spacewalk-list at redhat.com
> >https://www.redhat.com/mailman/listinfo/spacewalk-list
> >_______________________________________________
> >Spacewalk-list mailing list
> >Spacewalk-list at redhat.com
> >https://www.redhat.com/mailman/listinfo/spacewalk-list
> 
> cobbler not running??
> 
> 
> -- 
> sent from my mobile device