<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=Windows-1252"> <style type="text/css" style="display:none;"></style> </head> <body dir="ltr"> <div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;"> Basically you're doing everything that I've been doing with the exception of the db_recover command. I was not familiar with that command. How can I tell if the clients are self registering or not? Thank you. Daryl <div style="color: rgb(0, 0, 0);"> <div> <hr tabindex="-1" style="display:inline-block; width:98%"> <div id="x_divRplyFwdMsg" dir="ltr">From: Robert Paschedag <robert.paschedag@web.de> Sent: Friday, August 19, 2016 3:42 AM To: Daryl Rose Cc: spacewalk-list@redhat.com Subject: Re: [Spacewalk-list] Ongoing jabberd/osad issues. <div> </div> </div> </div> <div class="PlainText">Now I had also a problem with the database. Just wanted to check, which logfiles of the jabber db are not needed anymore as stated in <a href="http://web.stanford.edu/class/cs276a/projects/docs/berkeleydb/ref/transapp/logfile.html" id="LPlnk1817">http://web.stanford.edu/class/cs276a/projects/docs/berkeleydb/ref/transapp/logfile.html</a> <div id="LPBorder_GT_14716099508770.2547492988196607" style="margin-bottom: 20px; overflow: auto; width: 100%; text-indent: 0px;"> <table id="LPContainer_14716099508750.5177993408358306" cellspacing="0" style="width: 90%; position: relative; overflow: auto; padding-top: 20px; padding-bottom: 20px; margin-top: 20px; border-top: 1px dotted rgb(200, 200, 200); border-bottom: 1px dotted rgb(200, 200, 200); background-color: rgb(255, 255, 255);"> <tbody> <tr valign="top" style="border-spacing: 0px;"> <td id="TextCell_14716099508760.9327653146770483" colspan="2" style="vertical-align: top; position: relative; padding: 0px; display: table-cell;"> <div id="LPRemovePreviewContainer_14716099508760.5255006650074612"></div> <div id="LPTitle_14716099508760.9105873921020915" style="top: 0px; color: rgb(0, 120, 215); font-weight: normal; font-size: 21px; font-family: wf_segoe-ui_light, "Segoe UI Light", "Segoe WP Light", "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; line-height: 21px;"> <a id="LPUrlAnchor_14716099508760.4287264389348351" href="http://web.stanford.edu/class/cs276a/projects/docs/berkeleydb/ref/transapp/logfile.html" target="_blank" style="text-decoration: none;">Berkeley DB Reference Guide: Log file removal</a></div> <div id="LPMetadata_14716099508760.40954054911118787" style="margin: 10px 0px 16px; color: rgb(102, 102, 102); font-weight: normal; font-family: wf_segoe-ui_normal, "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; font-size: 14px; line-height: 14px;"> web.stanford.edu</div> <div id="LPDescription_14716099508770.6798984667494157" style="display: block; color: rgb(102, 102, 102); font-weight: normal; font-family: wf_segoe-ui_normal, "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; font-size: 14px; line-height: 20px; max-height: 100px; overflow: hidden;"> Log file removal. The fourth component of the infrastructure, log file removal, concerns the ongoing disk consumption of the database log files.</div> </td> </tr> </tbody> </table> </div> Just running "db_archive" killed my jabber db. And....fixing it with db_recover -v or db_recover -c within /var/lib/jabber/db did not work. So I was also in the situation to "clean" the database. 1. Stop jabberd (/etc/init.d/jabberd stop) 2. Stop osa-dispatcher (/etc/init.d/osa-dispatcher stop) 3. Remove contents of /var/lib/jabber/db (rm -f /var/lib/jabber/db/*) 4. Start jabber (/etc/init.d/jabberd start) 5. Start osa-dispatcher (/etc/init.d/osa-dispatcher start) I thought, I should restart the osad client everywhere....but no....the clients are just re-registering themeselves automatically. Of course, I have to check this on every client but what I have checked so far is looking good. Regards Robert Am 18.08.2016 um 09:30 schrieb Robert Paschedag: > Hi Daryl, > > as long as there are no error messages within the logs that there seems to be an error with the jabber db, I wouldn't do anything with the db. > > As I earlier wrote, I only had to repair the db once within about 3 1/2 years. > > So, what I would do now is to really delete the jabber db (back it up... just in case) to start up with a"clean " install. If the clients (that already have authentication information) do not re-register automatically, you should go to the client, stop osad, remove /etc/sysconfig/rhn/osad-auth.conf and start osad again. The client should then register and you should see the status on the web GUI as "online". If not, check the /var/log/rhn/osad.log on the client (if I remember correct right now) and osa-dispatcher logs in the server. > > I also wrote, that my spacewalk servers are NOT clients of themselves. I don't think, that should be a problem but just for " testing " you should deactivate osad "client" on the spacewalk server. > > Start with one test server. > > Good luck. > > Regards > Robert > Am 17.08.2016 20:43 schrieb Daryl Rose <darylrose@outlook.com>: >> >> I've posted here issues that I've had with jabberd and osad, as have others. But I haven't gotten things resolved, so I am posting additional information. >> >> >> I put SW into production about a year ago. After a period of time, I noticed issues with the WUI and servers not reporting correctly and other issues. Google searches show that I need to shutdown spacewalk and remove all the contents in /var/lib/jabberd/db. This seemed to work, but after a few months, I realized that osad was no longer communicating with osa-dispatcher. >> >> >> I started doing some additional research and learned that was not a good way to resolve this issue. According to the official Spacewalk documentation, I should create a checkpoint and then clean up log files keeping the database and auth database files. >> >> >> <a href="https://fedorahosted.org/spacewalk/wiki/JabberDatabase">https://fedorahosted.org/spacewalk/wiki/JabberDatabase</a> >> >> JabberDatabase – spacewalk - Fedora Hosted >> fedorahosted.org >> Jabber Database. Spacewalk utilizes Jabber to facilitate communications between the server and the clients for osa-dispatcher/osad. The Jabber program uses the ... >> These are the steps that I followed: >> >> >> /usr/bin/db_checkpoint -1 -h /var/lib/jabberd/db/ ## mark logs for deletion >> /usr/bin/db_archive -d -h /var/lib/jabberd/db/ ## delete logs >> service jabberd restart >> >> However, this also causes problems with jabberd and osad. If I use the commands as the documentation instructs, then osa-dispatcher will start, but die, and I get errors in the log that there is an invalid password. >> >> >> So to help explain my issue, I ran a test and tried to capture everything that I could and I'll post it here. >> >> >> 1. Listing of /var/lib/jabberd/db >> >> [root@<spwalk-server> db]# ls >> __db.001 __db.006 log.0000000004 log.0000000009 log.0000000014 log.0000000019 log.0000000024 sm.db >> __db.002 authreg.db log.0000000005 log.0000000010 log.0000000015 log.0000000020 log.0000000025 >> __db.003 log.0000000001 log.0000000006 log.0000000011 log.0000000016 log.0000000021 log.0000000026 >> __db.004 log.0000000002 log.0000000007 log.0000000012 log.0000000017 log.0000000022 log.0000000027 >> __db.005 log.0000000003 log.0000000008 log.0000000013 log.0000000018 log.0000000023 log.0000000028 >> >> 2. Spacewalk Server Status >> >> [root@<spwalk-server> db]# spacewalk-service status >> postmaster (pid 1175) is running... >> router (pid 21431) is running... >> sm (pid 21441) is running... >> c2s (pid 21451) is running... >> s2s (pid 21461) is running... >> tomcat6 (pid 1304) is running... [ OK ] >> httpd (pid 1385) is running... >> osa-dispatcher (pid 21479) is running... >> rhn-search is running (1441). >> cobblerd (pid 1491) is running... >> RHN Taskomatic is running (1515). >> >> 3. Most recent log file entry: >> >> 2016/08/17 07:44:13 -05:00 21476 0.0.0.0: osad/jabber_lib.__init__ >> 2016/08/17 07:44:13 -05:00 21476 0.0.0.0: osad/jabber_lib.setup_connection('Connected to jabber server', '<spwalk-server>.com') >> 2016/08/17 07:44:13 -05:00 21476 0.0.0.0: osad/osa_dispatcher.fix_connection('Upstream notification server started on port', 1290) >> 2016/08/17 07:44:14 -05:00 21476 0.0.0.0: osad/jabber_lib.process_forever >> >> 4. Ran the commands as instructed in the jabberd documentation. >> >> /usr/bin/db_checkpoint -1 -h /var/lib/jabberd/db/ ## mark logs for deletion >> /usr/bin/db_archive -d -h /var/lib/jabberd/db/ ## delete logs >> service jabberd restart >> >> 5. Log file entry: >> >> 2016/08/17 13:28:19 -05:00 21476 0.0.0.0: osad/jabber_lib.main('ERROR', 'Traceback (most recent call last):\n File "/usr/share/rhn/osad/jabber_lib.py", line 121, in main\n self.process_forever(c)\n File "/usr/share/rhn/osad/jabber_lib.py", line 179, in process_forever\n self.process_once(client)\n File "/usr/share/rhn/osad/osa_dispatcher.py", line 187, in process_once\n client.retrieve_roster()\n File "/usr/share/rhn/osad/jabber_lib.py", line 729, in retrieve_roster\n stanza = self.get_one_stanza()\n File "/usr/share/rhn/osad/jabber_lib.py", line 801, in get_one_stanza\n self.process(timeout=tm)\n File "/usr/share/rhn/osad/jabber_lib.py", line 1055, in process\n data = self._read(self.BLOCK_SIZE)\nSSLError: (\'OpenSSL error; will retry\', "(-1, \'Unexpected EOF\')")\n') >> 2016/08/17 13:28:29 -05:00 21476 0.0.0.0: osad/jabber_lib.__init__ >> 2016/08/17 13:28:29 -05:00 21476 0.0.0.0: osad/jabber_lib.setup_connection('Connected to jabber server', '<spwalk-server>.com') >> 2016/08/17 13:28:29 -05:00 21476 0.0.0.0: osad/jabber_lib.register('ERROR', 'Invalid password') >> >> 6. Spacewalk server status >> >> [root@<spwalk-server> db]# spacewalk-service status >> postmaster (pid 1175) is running... >> router (pid 27119) is running... >> sm (pid 27129) is running... >> c2s (pid 27139) is running... >> s2s (pid 27149) is running... >> tomcat6 (pid 1304) is running... [ OK ] >> httpd (pid 1385) is running... >> osa-dispatcher dead but pid file exists >> rhn-search is running (1441). >> cobblerd (pid 1491) is running... >> RHN Taskomatic is running (1515). >> >> 7. Long listing of /var/lib/jabberd/db >> >> [root@<spwalk-server> db]# ls -l >> total 7536 >> -rw-r-----. 1 jabber jabber 24576 Aug 17 13:28 __db.001 >> -rw-r-----. 1 jabber jabber 204800 Aug 17 13:29 __db.002 >> -rw-r-----. 1 jabber jabber 270336 Aug 17 13:29 __db.003 >> -rw-r-----. 1 jabber jabber 98304 Aug 17 13:29 __db.004 >> -rw-r-----. 1 jabber jabber 753664 Aug 17 13:29 __db.005 >> -rw-r-----. 1 jabber jabber 57344 Aug 17 13:29 __db.006 >> -rw-r-----. 1 jabber jabber 368640 Aug 17 07:46 authreg.db >> -rw-r-----. 1 jabber jabber 10485760 Aug 17 13:29 log.0000000031 >> -rw-r-----. 1 jabber jabber 487424 Aug 17 13:29 sm.db >> >> So, neither completely cleaning out jabberd database/log files works, and creating a checkpoint and removing log files that need to be cleaned out doesn't' work, so what can I do to get jabberd and osad to work, and to push out updates when I need to push them out? >> >> >> Thank you. >> >> >> Daryl >> >> >> >> >> >> >> >> >> >> >> >> >> > > _______________________________________________ > Spacewalk-list mailing list > Spacewalk-list@redhat.com > <a href="https://www.redhat.com/mailman/listinfo/spacewalk-list">https://www.redhat.com/mailman/listinfo/spacewalk-list</a> > </div> </div> </div> </body> </html>