[Spacewalk-list] Fwd: osa-dispatcher.service failed

Raymond Setchfield raymond.setchfield at gmail.com
Wed Nov 6 11:44:22 UTC 2019


Hi All,

Been testing this to see if I could potentially recreate the fault.

I have taken a copy of the instance which was working (vanilla install with
spacewalk applied). When the copy starts up. It appears to be loaded and
running fine. When I made a change to the server. For example, the
hostname. The OSAD failed after reboot.
I then made another machine from known working copy and changed the
sshd_config file. Rebooted the machine and osad then failed.again.

The failure seems to be the same, and there is nothing within the logs
which suggest why its failed. It appears to be a time out. anything else I
could potentially look into?

Thanks

Ray

---------- Forwarded message ---------
From: Raymond Setchfield <raymond.setchfield at gmail.com>
Date: Tue, Nov 5, 2019 at 10:00 AM
Subject: osa-dispatcher.service failed
To: <spacewalk-list at redhat.com>


Hi All,

I have been following this guide to install the SSL certificate, and it
appears to have worked fine. As I can see the certificate when logging into
spacewalk etc.

However, when restarting the services it appears that the osa-dispatcher
has failed. There is little or no information to go with to figure out why
it has failed.

● postgresql.service - PostgreSQL database server
   Loaded: loaded (/usr/lib/systemd/system/postgresql.service; enabled;
vendor preset: disabled)
   Active: active (running) since Tue 2019-11-05 09:50:15 UTC; 4min 41s ago
  Process: 12341 ExecStop=/usr/bin/pg_ctl stop -D ${PGDATA} -s -m fast
(code=exited, status=0/SUCCESS)
  Process: 12504 ExecStart=/usr/bin/pg_ctl start -D ${PGDATA} -s -o -p
${PGPORT} -w -t 300 (code=exited, status=0/SUCCESS)
  Process: 12487 ExecStartPre=/usr/bin/postgresql-check-db-dir ${PGDATA}
(code=exited, status=0/SUCCESS)
 Main PID: 12518 (postgres)
   CGroup: /system.slice/postgresql.service
           ├─12518 /usr/bin/postgres -D /var/lib/pgsql/data -p 5432
           ├─12529 postgres: logger process
           ├─12531 postgres: checkpointer process
           ├─12532 postgres: writer process
           ├─12533 postgres: wal writer process
           ├─12534 postgres: autovacuum launcher process
           ├─12535 postgres: stats collector process
           ├─12590 postgres: rhnuser rhnschema 127.0.0.1(44062) idle
           ├─12591 postgres: rhnuser rhnschema 127.0.0.1(44065) idle
           ├─12592 postgres: rhnuser rhnschema 127.0.0.1(44064) idle
           ├─12593 postgres: rhnuser rhnschema 127.0.0.1(44068) idle
           ├─12594 postgres: rhnuser rhnschema 127.0.0.1(44070) idle
           ├─12762 postgres: rhnuser rhnschema 127.0.0.1(44074) idle
           ├─12867 postgres: rhnuser rhnschema 127.0.0.1(44076) idle
           ├─12868 postgres: rhnuser rhnschema 127.0.0.1(44078) idle
           ├─12869 postgres: rhnuser rhnschema 127.0.0.1(44080) idle
           ├─12870 postgres: rhnuser rhnschema 127.0.0.1(44082) idle
           ├─12871 postgres: rhnuser rhnschema 127.0.0.1(44084) idle
           ├─12872 postgres: rhnuser rhnschema 127.0.0.1(44086) idle
           ├─12873 postgres: rhnuser rhnschema 127.0.0.1(44088) idle
           ├─12874 postgres: rhnuser rhnschema 127.0.0.1(44090) idle
           ├─12875 postgres: rhnuser rhnschema 127.0.0.1(44092) idle
           ├─12876 postgres: rhnuser rhnschema 127.0.0.1(44094) idle
           ├─12877 postgres: rhnuser rhnschema 127.0.0.1(44096) idle
           ├─12878 postgres: rhnuser rhnschema 127.0.0.1(44098) idle
           ├─12879 postgres: rhnuser rhnschema 127.0.0.1(44100) idle
           ├─12880 postgres: rhnuser rhnschema 127.0.0.1(44102) idle
           ├─12881 postgres: rhnuser rhnschema 127.0.0.1(44104) idle
           ├─12887 postgres: rhnuser rhnschema 127.0.0.1(44106) idle
           ├─12888 postgres: rhnuser rhnschema 127.0.0.1(44108) idle
           ├─12889 postgres: rhnuser rhnschema 127.0.0.1(44110) idle
           ├─12890 postgres: rhnuser rhnschema 127.0.0.1(44112) idle
           ├─12891 postgres: rhnuser rhnschema 127.0.0.1(44114) idle
           ├─12930 postgres: rhnuser rhnschema 127.0.0.1(44118) idle
           ├─12931 postgres: rhnuser rhnschema 127.0.0.1(44120) idle
           ├─12941 postgres: rhnuser rhnschema 127.0.0.1(44126) idle
           ├─12942 postgres: rhnuser rhnschema 127.0.0.1(44128) idle
           └─12943 postgres: rhnuser rhnschema 127.0.0.1(44130) idle

● jabberd.service - Jabber Server
   Loaded: loaded (/usr/lib/systemd/system/jabberd.service; enabled; vendor
preset: disabled)
   Active: active (exited) since Tue 2019-11-05 09:50:14 UTC; 4min 42s ago
  Process: 12486 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
 Main PID: 12486 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/jabberd.service

● tomcat.service - Apache Tomcat Web Application Container
   Loaded: loaded (/usr/lib/systemd/system/tomcat.service; enabled; vendor
preset: disabled)
   Active: active (running) since Tue 2019-11-05 09:50:14 UTC; 4min 42s ago
 Main PID: 12480 (java)
   CGroup: /system.slice/tomcat.service
           └─12480 /usr/lib/jvm/jre/bin/java -ea -Xms256m -Xmx256m
-Djava.awt.headless=true
-Dorg.xml.sax.driver=org.apache.xerces.parsers.SAXParser
-Dorg.apache.tomcat.util.htt...

● spacewalk-wait-for-tomcat.service - Spacewalk wait for tomcat
   Loaded: loaded
(/usr/lib/systemd/system/spacewalk-wait-for-tomcat.service; static; vendor
preset: disabled)
   Active: active (exited) since Tue 2019-11-05 09:50:31 UTC; 4min 25s ago
  Process: 12481 ExecStart=/usr/sbin/spacewalk-startup-helper
wait-for-tomcat (code=exited, status=0/SUCCESS)
 Main PID: 12481 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/spacewalk-wait-for-tomcat.service

● httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor
preset: disabled)
   Active: active (running) since Tue 2019-11-05 09:50:31 UTC; 4min 24s ago
     Docs: man:httpd(8)
           man:apachectl(8)
  Process: 12464 ExecStop=/bin/kill -WINCH ${MAINPID} (code=exited,
status=0/SUCCESS)
 Main PID: 12676 (httpd)
   Status: "Total requests: 13; Current requests/sec: 0; Current traffic:
0 B/sec"
   CGroup: /system.slice/httpd.service
           ├─12676 /usr/sbin/httpd -DFOREGROUND
           ├─12677 /usr/sbin/httpd -DFOREGROUND
           ├─12678 /usr/sbin/httpd -DFOREGROUND
           ├─12680 /usr/sbin/httpd -DFOREGROUND
           ├─12681 /usr/sbin/httpd -DFOREGROUND
           ├─12683 /usr/sbin/httpd -DFOREGROUND
           ├─12685 /usr/sbin/httpd -DFOREGROUND
           ├─12686 /usr/sbin/httpd -DFOREGROUND
           └─12687 /usr/sbin/httpd -DFOREGROUND

● spacewalk-wait-for-jabberd.service - Spacewalk wait for jabberd
   Loaded: loaded
(/usr/lib/systemd/system/spacewalk-wait-for-jabberd.service; static; vendor
preset: disabled)
   Active: active (exited) since Tue 2019-11-05 09:50:14 UTC; 4min 42s ago
  Process: 12490 ExecStart=/usr/sbin/spacewalk-startup-helper
wait-for-jabberd (code=exited, status=0/SUCCESS)
 Main PID: 12490 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/spacewalk-wait-for-jabberd.service

● osa-dispatcher.service - OSA Dispatcher daemon
   Loaded: loaded (/usr/lib/systemd/system/osa-dispatcher.service; enabled;
vendor preset: disabled)
   Active: failed (Result: timeout) since Tue 2019-11-05 09:51:45 UTC; 3min
11s ago
  Process: 12544 ExecStart=/usr/sbin/osa-dispatcher --pid-file
/var/run/osa-dispatcher.pid (code=killed, signal=TERM)
  Process: 12542 ExecStartPre=/bin/rm -f /var/run/osa-dispatcher.pid
(code=exited, status=0/SUCCESS)

● rhn-search.service - Spacewalk search engine
   Loaded: loaded (/usr/lib/systemd/system/rhn-search.service; enabled;
vendor preset: disabled)
   Active: active (running) since Tue 2019-11-05 09:50:32 UTC; 4min 24s ago
  Process: 12350 ExecStop=/usr/sbin/rhn-search stop (code=exited,
status=0/SUCCESS)
  Process: 12679 ExecStart=/usr/sbin/rhn-search start (code=exited,
status=0/SUCCESS)
 Main PID: 12703 (rhnsearchd)
   CGroup: /system.slice/rhn-search.service
           ├─12703 /usr/bin/rhnsearchd
/usr/share/rhn/config-defaults/rhn_search_daemon.conf
wrapper.pidfile=/var/run/rhn-search.pid wrapper.daemonize=TRUE
           └─12716 /usr/bin/java
-Djava.library.path=/usr/lib:/usr/lib64:/usr/lib/oracle/11.2/client/lib:/usr/lib/oracle/11.2/client64/lib:/usr/lib/gcj/postgresql-jdbc:/usr/lib6...

● cobblerd.service - Cobbler daemon
   Loaded: loaded (/usr/lib/systemd/system/cobblerd.service; enabled;
vendor preset: disabled)
   Active: active (running) since Tue 2019-11-05 09:50:31 UTC; 4min 24s ago
 Main PID: 12682 (cobblerd)
   CGroup: /system.slice/cobblerd.service
           └─12682 /usr/bin/python2 -s /usr/bin/cobblerd --no-daemonize

● taskomatic.service - Taskomatic
   Loaded: loaded (/usr/lib/systemd/system/taskomatic.service; enabled;
vendor preset: disabled)
   Active: active (running) since Tue 2019-11-05 09:50:32 UTC; 4min 24s ago
  Process: 12348 ExecStop=/usr/sbin/taskomatic stop (code=exited,
status=0/SUCCESS)
  Process: 12684 ExecStart=/usr/sbin/taskomatic start (code=exited,
status=0/SUCCESS)
 Main PID: 12714 (taskomaticd)
   CGroup: /system.slice/taskomatic.service
           ├─12714 /usr/bin/taskomaticd
/usr/share/rhn/config-defaults/rhn_taskomatic_daemon.conf
wrapper.pidfile=/var/run//taskomatic.pid wrapper.daemonize=TRUE
           └─12720 /usr/bin/java -Dibm.dst.compatibility=true
-Dfile.encoding=UTF-8 -Xms256m -Xmx1024m
-Djava.library.path=/usr/lib:/usr/lib64:/usr/lib/oracle/11.2/client64/lib:...
● spacewalk.target - Spacewalk
   Loaded: loaded (/usr/lib/systemd/system/spacewalk.target; enabled;
vendor preset: disabled)
   Active: inactive (dead)

[root at smart-cent-mgmt ~]# tail -f /var/log/rhn/osa-dispatcher.log
2019/11/04 15:43:47 -00:00 1160 0.0.0.0: osad/jabber_lib.__init__
2019/11/05 08:49:32 -00:00 7952 0.0.0.0: osad/jabber_lib.__init__
2019/11/05 08:53:00 -00:00 8449 0.0.0.0: osad/jabber_lib.__init__
2019/11/05 08:58:38 -00:00 8738 0.0.0.0: osad/jabber_lib.__init__
2019/11/05 09:14:49 -00:00 9621 0.0.0.0: osad/jabber_lib.__init__
2019/11/05 09:19:31 -00:00 10318 0.0.0.0: osad/jabber_lib.__init__
2019/11/05 09:25:15 -00:00 11420 0.0.0.0: osad/jabber_lib.__init__
2019/11/05 09:37:06 -00:00 12049 0.0.0.0: osad/jabber_lib.__init__
2019/11/05 09:43:39 -00:00 12162 0.0.0.0: osad/jabber_lib.__init__
2019/11/05 09:50:15 -00:00 12544 0.0.0.0: osad/jabber_lib.__init__

Nov 05 09:51:45 cent-mgmt.domain.com systemd[1]: osa-dispatcher.service
start operation timed out. Terminating.
Nov 05 09:51:45 cent-mgmt.domain.com systemd[1]: Failed to start OSA
Dispatcher daemon.
-- Subject: Unit osa-dispatcher.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit osa-dispatcher.service has failed.
--
-- The result is failed.
Nov 05 09:51:45 cent-mgmt.domain.com systemd[1]: Dependency failed for
Spacewalk.
-- Subject: Unit spacewalk.target has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit spacewalk.target has failed.
--
-- The result is dependency.
Nov 05 09:51:45 cent-mgmt.domain.com systemd[1]: Job spacewalk.target/start
failed with result 'dependency'.
Nov 05 09:51:45 cent-mgmt.domain.com systemd[1]: Unit
osa-dispatcher.service entered failed state.
Nov 05 09:51:45 cent-mgmt.domain.com systemd[1]: osa-dispatcher.service
failed.
Nov 05 09:51:45 cent-mgmt.domain.com polkitd[653]: Unregistered
Authentication Agent for unix-process:12474:6519546 (system bus name
:1.133, object path /org/freedesktop/PolicyKit1/Au

Can any please advise? The only thing which I can think of is a change in
hostname which took place before this, and this was ran using
"spacewalk-hostname-rename" command

Thanks

Ray
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/spacewalk-list/attachments/20191106/ba5b99f7/attachment.htm>


More information about the Spacewalk-list mailing list