[Spacewalk-list] Spacewalk 1.7 / RHEL 5.8 / Taskomatic issue

Thu Jul 19 10:30:55 UTC 2012

Hello,

thanks for feedback.

First of all I'd like to notice, that those pieces of logs I put onto my initial email WERE actually pieces of taskomatic logs - I surely wouldn't put any sort of random logs :)

Anyway also - I got this sorted. However those taskomatic / jvm pieces of logs confused me a bit I finally found the source of my issue. Something that I missed apparently during my installation - that it's the best to keep spacewalk as only virtual host, unless we want to rewrite a lot of configs ( and still not sure if it would be completely ok ).

The thing is I wanted to use another virtual host for something else on the same box - and it apparently confused spacewalk/taskomatic installation, so taskomatic couldn't connect properly to do its job.

So - luckily it turned out to be a small thing - but a bit painful in a first moment and took me quite a while to understand what's the problem.

Lesson taken.

So the issue is over - but I wanted to put the info about how I fixed it in case somehow has similar issue, trying to use multiple virtual hosts on spacewalk system (not a very good idea apparently :))

Regards,
Jacek Rudowski
jrudowski at sapphire.gi
DDI: +350 20049785
Sapphire Networks
Suite 3.0.3, Eurotowers, P O Box 797, Gibraltar 
Customer Services
( Tel: +350 20047200
F Fax: +350 20047272
* E-Mail: info at sapphire.gi     
: Web: http://www.sapphire.gi 

Technical Support
( Tel: +350 20047201
F Fax: +350 20047271
* E-Mail: support at sapphire.gi 

-----Original Message-----
From: Jan Hutař [mailto:jhutar at redhat.com] 
Sent: 13 July 2012 17:44
To: spacewalk-list at redhat.com
Cc: Jacek Rudowski
Subject: Re: [Spacewalk-list] Spacewalk 1.7 / RHEL 5.8 / Taskomatic issue

On Thu, 5 Jul 2012 16:06:54 +0000 Jacek Rudowski
<jrudowski at sapphire.gi> wrote:

> Hello everybody,
> 
> I hope someone will be able to help me with my problems with
> taskomatic.
> 
> To be as descriptive as possible and give as much information
> as possible - a bit of background...
> 
> First installation of Spacewalk 1.7 has been done on RHEL 5.5
> and I have to say it started to work almost instantly (with
> exception for cobbler, which had to be downgraded due to some
> apparently known issues - but that's not the case here)
> 
> I connected it with mrepo generated repositories, with
> packages synced nightly to spacewalk, and also with taskomatic
> running fine - so I had also repo files prepared daily for
> spacewalk and rhnplugin.
> 
> Later I have upgraded RHEL to 5.8 and... taskomatic stopped
> working, and to be honest I have no idea why, which change in
> the system it might be related to.
> 
> repomd.xml is not generated anymore for spacewalk/rhnplugin
> and taskomatic dying a while after (re)start attempt.
> 
> With some debug logging turned on I find those:
> 
> 
> ERROR  | wrapper  | 2012/07/04 10:24:45 | Startup failed:
> Timed out waiting for signal from JVM. ERROR  | wrapper  |
> 2012/07/04 10:24:45 | JVM did not exit on request, terminated
> DEBUG  | wrapper  | 2012/07/04 10:24:45 | Signal trapped.
> Details: DEBUG  | wrapper  | 2012/07/04 10:24:45 |   signal
> number=17 (SIGCHLD), source="unknown" DEBUG  | wrapper  |
> 2012/07/04 10:24:45 | Received SIGCHLD, checking JVM process
> status. DEBUG  | wrapperp | 2012/07/04 10:24:45 | socket read
> no code (closed?). DEBUG  | wrapperp | 2012/07/04 10:24:45 |
> Closing backend socket.
> 
> I was googling for information on this - it looks to be quite
> a common issue with earlier versions of spacewalk, but also
> expected to be sorted(?)
> 
> Anyway - some further investigation, with even more logging
> gave me this:
> 
> INFO   | jvm 1    | 2012/07/04 16:41:07 | 2012-07-04
> 16:41:07,180 [WrapperListener_start_runner] INFO
> com.redhat.rhn.taskomatic.core.TaskomaticDaemon - Tomcat is
> not up yet, sleeping 4 seconds INFO   | jvm 1    | 2012/07/04
> 16:41:11 | 2012-07-04 16:41:11,187
> [WrapperListener_start_runner] INFO
> com.redhat.rhn.taskomatic.core.TaskomaticDaemon - Tomcat is
> not up yet, sleeping 4 seconds INFO   | jvm 1    | 2012/07/04
> 16:41:15 | 2012-07-04 16:41:15,193
> [WrapperListener_start_runner] INFO
> com.redhat.rhn.taskomatic.core.TaskomaticDaemon - Tomcat is
> not up yet, sleeping 4 seconds INFO   | jvm 1    | 2012/07/04
> 16:41:19 | 2012-07-04 16:41:19,199
> [WrapperListener_start_runner] INFO
> com.redhat.rhn.taskomatic.core.TaskomaticDaemon - Tomcat is
> not up yet, sleeping 4 seconds INFO   | jvm 1    | 2012/07/04
> 16:41:23 | 2012-07-04 16:41:23,205
> [WrapperListener_start_runner] INFO
> com.redhat.rhn.taskomatic.core.TaskomaticDaemon - Tomcat is
> not up yet, sleeping 4 seconds INFO   | jvm 1    | 2012/07/04
> 16:41:27 | 2012-07-04 16:41:27,212
> [WrapperListener_start_runner] INFO
> com.redhat.rhn.taskomatic.core.TaskomaticDaemon - Tomcat is
> not up yet, sleeping 4 seconds
> 
> But... tomcat is running and listening on port 8009 for ajp
> proxied connections (which I think is default in spacewalk
> installation)
> 
> 
> I don't really know now where too look at - maybe someone
> experienced similar sort of issue already?
> 
> One more thing maybe - from catalina.out:
> 
> Jul 5, 2012 5:11:27 PM org.apache.coyote.ajp.AjpMessage
> processHeader SEVERE: Invalid message recieved with signature
> 18245
> 
> But it doesn't really help me much - maybe because of lack of
> some skills in Java (couple of more weeks with those issues,
> and it will probably change... ha ha!)
> 
> Just yet couple of word about environment:
> as mentioned earlier RHEL 5.8
> Spacewalk 1.7
> TanukiWrapper 3.2.1
> Java 1.6.0  (tried OpenJDK, Sun, IBM - no luck - now it's Sun)
> 
> Any suggestions would be welcome - any questions for
> additional information either, of course...
> 
> Regards

Really not a knowledgeable person here, but as nobody answered
yet, could you try to start taskomatic manually:

# service taskomatic stop   # probably useless in your case :)
# /usr/bin/taskomaticd /usr/share/rhn/config-defaults/rhn_taskomatic_daemon.conf
wrapper  | --> Wrapper Started as Console
wrapper  | Launching a JVM...
jvm 1    | Wrapper (Version 3.2.1) http://wrapper.tanukisoftware.org
jvm 1    |

And give these lists after `rhn-satellite restart`:

# rpm -qa | grep java | sort
# ps ax | grep java

Maybe this will provide more clues?

Regards,
Jan

-- 
Jan Hutar     Systems Management QA
jhutar at redhat.com     Red Hat, Inc.