[Spacewalk-list] Taskomatic won't start in 0.8 and 1.0
Brian T. O'Neill
btoneill at walleyesoftware.com
Tue May 18 15:45:06 UTC 2010
This could be the same issue I'm having. Take a look at
/var/log/httpd/error_log and see if you have:
[Mon May 17 16:32:02 2010] [error] [client 127.0.0.1] File does not
exist: /var/www/html/rpc
They would be showing up at 4 second intervals during each of the 5
attempts that it tries to start taskomatic.
If it is, sounds like you and I have the same problem of /rpc/api not
working over http.
Brian
-----Original Message-----
From: spacewalk-list-bounces at redhat.com
[mailto:spacewalk-list-bounces at redhat.com] On Behalf Of Justin Sherrill
Sent: Tuesday, May 18, 2010 11:31 AM
To: spacewalk-list at redhat.com
Subject: Re: [Spacewalk-list] Taskomatic won't start in 0.8 and 1.0
On 5/17/10 1:45 PM, Mike Hanby wrote:
> Figured I'd update the thread with my latest tests:
>
> I stopped all SW processes and renamed /var/cache/rhn, created a new
/var/cache/rhn giving apache ownership and 750 permissions.
>
> Started everything back up and taskomatic still fails to start up
properly.
>
> Is there any way to get debugging/logging from the JVM itself, or does
the wrapper provide the only logging?
>
> I also tried to increasing the ping timeout, but that also did not
result in any different behavior.
>
> Mike
>
> -----Original Message-----
> From: spacewalk-list-bounces at redhat.com
[mailto:spacewalk-list-bounces at redhat.com] On Behalf Of Mike Hanby
> Sent: Friday, May 14, 2010 3:43 PM
> To: spacewalk-list at redhat.com
> Subject: Re: [Spacewalk-list] Taskomatic won't start in 0.8 and 1.0
>
> Here's an example of my EPEL 5 x86_64 channel details page:
> Last Modified: 2010-05-14 05:16:39 EDT
> Last Repo Build: 2010-04-21 18:05:08 EDT
>
> Showing that spacewalk-repo-sync inserted new packages this morning,
yet the repo build is still very old.
>
> -----Original Message-----
> From: spacewalk-list-bounces at redhat.com
[mailto:spacewalk-list-bounces at redhat.com] On Behalf Of Mike Hanby
> Sent: Friday, May 14, 2010 3:24 PM
> To: spacewalk-list at redhat.com
> Subject: [Spacewalk-list] Taskomatic won't start in 0.8 and 1.0
>
> Howdy,
>
> The last time any of my /var/cache/rhn/repodata was updated was Apr
23. This was also the day I upgraded to 0.8 from 0.7 AND ported my
database from OracleXE to Oracle 11gR2.
>
> Everything else appears to be working. My nightly
/usr/bin/spacewalk-repo-sync successfully pulls updated packages from
the various public mirrors and inserts them into the database. I can
browse the Spacewalk web interface and perform searches to see the
latest packages and my SW clients can all download packages via yum from
the spacewalk server. The catch there is, only packages that were part
of the system Apr 23 and prior since the repodata is old.
>
> I enabled DEBUG in the rhn_taskomatic_daemon.conf file for console and
started taskomatic to gather the output:
>
> # /etc/init.d/taskomatic console
> Running RHN Taskomatic...
> wrapper | --> Wrapper Started as Console
> wrapper | Using tick timer.
> wrapperp | server listening on port 32001.
> wrapper | Command[0] : /usr/bin/java
> wrapper | Command[1] : -Dibm.dst.compatibility=true
> wrapper | Command[2] : -Xms256m
> wrapper | Command[3] : -Xmx512m
> wrapper | Command[4] : -Djava.library.path=/usr/lib:/usr/lib64
> wrapper | Command[5] : -classpath
> wrapper | Command[6] :
/usr/share/java/tanukiwrapper.jar:/usr/share/rhn/classes:/usr/share/java
/struts.jar:/usr/share/java/jfreechart.jar:/usr/share/java/jpam.jar:/usr
/share/java/javamail.jar:/usr/share/java/axis/axis-ant.jar:/usr/share/ja
va/quartz.jar:/usr/share/java/commons-codec.jar:/usr/share/java/commons-
beanutils.jar:/usr/share/java/ojdbc14.jar:/usr/share/java/jta.jar:/usr/s
hare/java/concurrent.jar:/usr/share/java/asm/asm.jar:/usr/share/java/axi
s/jaxrpc.jar:/usr/share/java/commons-collections.jar:/usr/share/java/tag
libs-standard.jar:/usr/share/java/axis/axis.jar:/usr/share/java/xalan-j2
.jar:/usr/share/java/commons-validator.jar:/usr/share/java/asm/asm-attrs
.jar:/usr/share/java/jaf.jar:/usr/share/java/jdom.jar:/usr/share/java/or
o.jar:/usr/share/rhn/lib/rhn.jar:/usr/share/java/redstone-xmlrpc.jar:/us
r/share/java/oscache.jar:/usr/share/java/log4j.jar:/usr/share/java/wsdl4
j.jar:/usr/share/java/jcommon.jar:/usr/share/java/commons-el.jar:/usr/sh
are/java/taglibs-core.jar:/u
s!
>
r/share/java/commons-lang.jar:/usr/share/java/commons-digester.jar:/usr/
share/java/jasper5-runtime.jar:/usr/share/java/jspapi.jar:/usr/share/jav
a/commons-fileupload.jar:/usr/share/java/c3p0.jar:/usr/share/java/siteme
sh.jar:/usr/share/java/jasper5-compiler.jar:/usr/share/java/axis/saaj.ja
r:/usr/share/java/commons-logging.jar:/usr/share/java/commons-discovery.
jar:/usr/share/java/xml-commons-apis.jar:/usr/share/java/axis/jaxrpc.jar
:/usr/share/java/commons-cli.jar:/usr/share/java/bcel.jar:/usr/share/jav
a/antlr.jar:/usr/share/java/xerces-j2.jar:/usr/share/java/hibernate3.jar
:/usr/share/java/cglib.jar:/usr/share/java/dom4j.jar:/usr/share/rhn/lib/
java-branding.jar
> wrapper | Command[7] : -Dwrapper.key=gR3LlUzzdxJT0EFJ
> wrapper | Command[8] : -Dwrapper.port=32001
> wrapper | Command[9] : -Dwrapper.jvm.port.min=31000
> wrapper | Command[10] : -Dwrapper.jvm.port.max=31999
> wrapper | Command[11] : -Dwrapper.debug=TRUE
> wrapper | Command[12] : -Dwrapper.pid=6329
> wrapper | Command[13] : -Dwrapper.version=3.2.1
> wrapper | Command[14] : -Dwrapper.native_library=wrapper
> wrapper | Command[15] : -Dwrapper.cpu.timeout=10
> wrapper | Command[16] : -Dwrapper.jvmid=1
> wrapper | Command[17] :
com.redhat.rhn.taskomatic.core.TaskomaticDaemon
> wrapper | Launching a JVM...
> jvm 1 | WrapperManager class initialized by thread: main Using
classloader: sun.misc.Launcher$AppClassLoader at 64601bb1
> jvm 1 | Wrapper (Version 3.2.1) http://wrapper.tanukisoftware.org
> jvm 1 |
> jvm 1 | Wrapper Manager: JVM #1
> jvm 1 | Running a 64-bit JVM.
> jvm 1 | Wrapper Manager: Registering shutdown hook
> jvm 1 | Wrapper Manager: Using wrapper
> jvm 1 | Load native library. One or more attempts may fail if
platform specific libraries do not exist.
> jvm 1 | Loading native library failed: libwrapper-linux-x86-64.so
Cause: java.lang.UnsatisfiedLinkError: no wrapper-linux-x86-64 in
java.library.path
> jvm 1 | Loaded native library: libwrapper.so
> jvm 1 | Calling native initialization method.
> jvm 1 | Inside native WrapperManager initialization method
> jvm 1 | Java Version : 1.6.0-b09 OpenJDK 64-Bit Server VM
> jvm 1 | Java VM Vendor : Sun Microsystems Inc.
> jvm 1 |
> jvm 1 |
WrapperManager.start(com.redhat.rhn.taskomatic.core.TaskomaticDaemon at 348
7a5cc, args[]) called by thread: main
> jvm 1 | Control event monitor thread started.
> jvm 1 | Startup runner thread started.
> jvm 1 | Communications runner thread started.
> jvm 1 | Open socket to wrapper...Wrapper-Connection
> jvm 1 | Failed attempt to bind using local port 31000
> jvm 1 | Opened Socket from 31001 to 32001
> jvm 1 | Send a packet KEY : gR3LlUzzdxJT0EFJ
> jvm 1 |
handleSocket(Socket[addr=/127.0.0.1,port=32001,localport=31001])
> wrapperp | accepted a socket from 127.0.0.1 on port 31001
> wrapperp | read a packet KEY : gR3LlUzzdxJT0EFJ
> wrapper | Got key from JVM: gR3LlUzzdxJT0EFJ
> wrapperp | send a packet LOW_LOG_LEVEL : 1
> wrapperp | send a packet PING_TIMEOUT : 30
> wrapperp | send a packet PROPERTIES : (Property Values)
> wrapper | Start Application.
> wrapperp | send a packet START : start
> jvm 1 | Received a packet LOW_LOG_LEVEL : 1
> jvm 1 | Wrapper Manager: LowLogLevel from Wrapper is 1
> jvm 1 | Received a packet PING_TIMEOUT : 30
> jvm 1 | PingTimeout from Wrapper is 30000
> jvm 1 | Received a packet PROPERTIES : (Property Values)
> jvm 1 | Received a packet START : start
> jvm 1 | calling WrapperListener.start()
> jvm 1 | Waiting for WrapperListener.start runner thread to
complete.
> jvm 1 | WrapperListener.start runner thread started.
> wrapper | Startup failed: Timed out waiting for signal from JVM.
> wrapper | JVM did not exit on request, terminated
> wrapper | Signal trapped. Details:
> wrapper | signal number=17 (SIGCHLD), source="unknown"
> wrapper | Received SIGCHLD, calling wait().
> wrapper | wait() returned, child process should be gone.
> wrapperp | server listening on port 32001.
> wrapper | JVM was only running for 29 seconds leading to a failed
restart count of 1.
> wrapper | Waiting 5 seconds before launching another JVM.
> wrapper | Command[0] : /usr/bin/java
> wrapper | Command[1] : -Dibm.dst.compatibility=true
> wrapper | Command[2] : -Xms256m
> wrapper | Command[3] : -Xmx512m
> wrapper | Command[4] : -Djava.library.path=/usr/lib:/usr/lib64
> wrapper | Command[5] : -classpath
> ... repeats thru jvm 5 ...
> jvm 5 | Received a packet PROPERTIES : (Property Values)
> jvm 5 | Received a packet START : start
> jvm 5 | calling WrapperListener.start()
> jvm 5 | Waiting for WrapperListener.start runner thread to
complete.
> jvm 5 | WrapperListener.start runner thread started.
> wrapper | Startup failed: Timed out waiting for signal from JVM.
> wrapper | JVM did not exit on request, terminated
> wrapper | Signal trapped. Details:
> wrapper | signal number=17 (SIGCHLD), source="unknown"
> wrapper | Received SIGCHLD, calling wait().
> wrapper | wait() returned, child process should be gone.
> wrapperp | server listening on port 32001.
> wrapper | JVM was only running for 29 seconds leading to a failed
restart count of 5.
> wrapper | There were 5 failed launches in a row, each lasting less
than 300 seconds. Giving up.
> wrapper | There may be a configuration problem: please check the
logs.
> wrapper | <-- Wrapper Stopped
>
>
> I tried deleting the contents of the rhnTaskQueue, but the taskomatic
service still fails to start
> $ sqlpus '/ as sysdba'
> SQL> delete from SPACEWALK.rhnTaskQueue;
>
> 20 rows deleted.
>
> SQL> select * from SPACEWALK.rhnTaskQueue;
>
> no rows selected
>
>
> Anyone have any suggestions on what to try next? While I debug this
issue, is there a way from the command line to force the repodata
creation for each channel?
>
> Thanks, Mike
Hey Mike,
Someone was on #spaceawlk the other day with this exact problem that we
couldn't figure out. Was it you (just curious)?
I'd try this:
edit /usr/share/rhn/classes/log4j.properties
and add:
log4j.logger.com.redhat.rhn.taskomatic=DEBUG
Then restart taskomatic and monitor /var/log/rhn/rhn_taskomatic.log
Also, can you upload your /etc/rhn/default/rhn_taskomatic* files?
Thanks
-Justin Sherrill
>
> =================================
> Mike Hanby
> mhanby at uab.edu
> Information Systems Specialist II
> IT HPCS / Research Computing
>
>
>
> _______________________________________________
> Spacewalk-list mailing list
> Spacewalk-list at redhat.com
> https://www.redhat.com/mailman/listinfo/spacewalk-list
>
> _______________________________________________
> Spacewalk-list mailing list
> Spacewalk-list at redhat.com
> https://www.redhat.com/mailman/listinfo/spacewalk-list
>
> _______________________________________________
> Spacewalk-list mailing list
> Spacewalk-list at redhat.com
> https://www.redhat.com/mailman/listinfo/spacewalk-list
--
Justin Sherrill, RHCA 1801 Varsity Drive.
Software Engineer Raleigh, NC 27603
Red Hat, Inc.
_______________________________________________
Spacewalk-list mailing list
Spacewalk-list at redhat.com
https://www.redhat.com/mailman/listinfo/spacewalk-list
More information about the Spacewalk-list
mailing list