[Spacewalk-list] Spacewalk issue with deleting profiles, listing kickstarts, and the cobblersynctask

Greason, Michael R. (GSFC-670.0)[ADNET SYSTEMS INC] michael.r.greason at nasa.gov
Wed Jun 20 21:39:08 UTC 2018


I’ve an existing Spacewalk 2.6 installation (running on CentOS 7.5)  and a new Spacewalk 2.8 installation (originally on CentoOS 7.5 but now on CentOS 6.9); both are throwing Internal server errors that I cannot explain or figure out. Whenever I attempt to delete a profile (only in the older installation; the new one doesn’t have any systems assigned yet) or attempt to list kickstart profiles, I see an internal server error. I see a complaint in the tomcat logs about not being able to log in. When the CobblerSyncTask runs (every minute), I see a similar error in the rhn_taskomatic logs.  I include samples below.

The 2.8 installation was originally on a CentOS 7.5 system but I found Googled reference implying that there was a problem with this so I downgraded. This is a new system; I have a collection of spacecmd commands to create channels, organizations, users, system groups, and the like but have not assigned any systems to the latest attempt as yet. I’m running into a separate certificate authority issue with spacewalk-repo-sync and so will be rebuilding this system yet again with CentOS 7.5 but am sure I’ll run into the same problem and so thought I’d go ahead and ask for help now.

The 2.6 system developed this problem some time in the last six months; one of my organizations asked to use kickstart profiles and initially they worked. Unfortunately I cannot put a date on when it started to break down.

Thank you,
Michael Greason

CobblerSyncTask logs
Rhn_taskomatic_daemon.log excerpt
INFO   | jvm 1    | 2018/06/20 17:12:00 | 2018-06-20 17:12:00,207 [DefaultQuartzScheduler_Worker-10] ERROR com.redhat.rhn.manager.kickstart.cobbler.CobblerLoginCommand - XmlRpcFault while logging in.  most likely user doesn't have permissions.
INFO   | jvm 1    | 2018/06/20 17:12:00 | redstone.xmlrpc.XmlRpcFault: <class 'cobbler.cexceptions.CX'>:'login failed (taskomatic_user)'
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at redstone.xmlrpc.XmlRpcClient.handleResponse(XmlRpcClient.java:443)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at redstone.xmlrpc.XmlRpcClient.endCall(XmlRpcClient.java:376)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at redstone.xmlrpc.XmlRpcClient.invoke(XmlRpcClient.java:165)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.manager.kickstart.cobbler.CobblerXMLRPCHelper.invokeMethod(CobblerXMLRPCHelper.java:69)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.manager.kickstart.cobbler.CobblerLoginCommand.login(CobblerLoginCommand.java:52)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.frontend.integration.IntegrationService.authorize(IntegrationService.java:114)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.frontend.integration.IntegrationService.getAuthToken(IntegrationService.java:67)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.manager.kickstart.cobbler.CobblerCommand.<init>(CobblerCommand.java:68)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.manager.kickstart.cobbler.CobblerDistroSyncCommand.<init>(CobblerDistroSyncCommand.java:52)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.taskomatic.task.CobblerSyncTask.execute(CobblerSyncTask.java:78)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.taskomatic.task.RhnJavaJob.execute(RhnJavaJob.java:88)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.taskomatic.TaskoJob.execute(TaskoJob.java:186)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
INFO   | jvm 1    | 2018/06/20 17:12:00 | 2018-06-20 17:12:00,207 [DefaultQuartzScheduler_Worker-10] ERROR com.redhat.rhn.taskomatic.task.CobblerSyncTask - RuntimeExceptionError trying to sync to cobbler: We had an error trying to login.
INFO   | jvm 1    | 2018/06/20 17:12:00 | com.redhat.rhn.manager.kickstart.cobbler.NoCobblerTokenException: We had an error trying to login.
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.manager.kickstart.cobbler.CobblerLoginCommand.login(CobblerLoginCommand.java:57)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.frontend.integration.IntegrationService.authorize(IntegrationService.java:114)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.frontend.integration.IntegrationService.getAuthToken(IntegrationService.java:67)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.manager.kickstart.cobbler.CobblerCommand.<init>(CobblerCommand.java:68)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.manager.kickstart.cobbler.CobblerDistroSyncCommand.<init>(CobblerDistroSyncCommand.java:52)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.taskomatic.task.CobblerSyncTask.execute(CobblerSyncTask.java:78)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.taskomatic.task.RhnJavaJob.execute(RhnJavaJob.java:88)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.taskomatic.TaskoJob.execute(TaskoJob.java:186)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
INFO   | jvm 1    | 2018/06/20 17:12:00 | Caused by: redstone.xmlrpc.XmlRpcFault: <class 'cobbler.cexceptions.CX'>:'login failed (taskomatic_user)'
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at redstone.xmlrpc.XmlRpcClient.handleResponse(XmlRpcClient.java:443)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at redstone.xmlrpc.XmlRpcClient.endCall(XmlRpcClient.java:376)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at redstone.xmlrpc.XmlRpcClient.invoke(XmlRpcClient.java:165)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.manager.kickstart.cobbler.CobblerXMLRPCHelper.invokeMethod(CobblerXMLRPCHelper.java:69)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.manager.kickstart.cobbler.CobblerLoginCommand.login(CobblerLoginCommand.java:52)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               ... 9 more
INFO   | jvm 1    | 2018/06/20 17:12:00 | 2018-06-20 17:12:00,208 [DefaultQuartzScheduler_Worker-10] ERROR com.redhat.rhn.taskomatic.task.CobblerSyncTask - re-throwing exception since we havent yet.
INFO   | jvm 1    | 2018/06/20 17:12:00 | 2018-06-20 17:12:00,209 [DefaultQuartzScheduler_Worker-10] ERROR com.redhat.rhn.taskomatic.task.CobblerSyncTask - Executing a task threw an exception: com.redhat.rhn.manager.kickstart.cobbler.NoCobblerTokenException
INFO   | jvm 1    | 2018/06/20 17:12:00 | 2018-06-20 17:12:00,209 [DefaultQuartzScheduler_Worker-10] ERROR com.redhat.rhn.taskomatic.task.CobblerSyncTask - Message: We had an error trying to login.
INFO   | jvm 1    | 2018/06/20 17:12:00 | 2018-06-20 17:12:00,209 [DefaultQuartzScheduler_Worker-10] ERROR com.redhat.rhn.taskomatic.task.CobblerSyncTask - Cause: redstone.xmlrpc.XmlRpcFault: <class 'cobbler.cexceptions.CX'>:'login failed (taskomatic_user)'
INFO   | jvm 1    | 2018/06/20 17:12:00 | 2018-06-20 17:12:00,209 [DefaultQuartzScheduler_Worker-10] ERROR com.redhat.rhn.taskomatic.task.CobblerSyncTask - Stack trace:com.redhat.rhn.manager.kickstart.cobbler.NoCobblerTokenException: We had an error trying to login.
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.manager.kickstart.cobbler.CobblerLoginCommand.login(CobblerLoginCommand.java:57)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.frontend.integration.IntegrationService.authorize(IntegrationService.java:114)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.frontend.integration.IntegrationService.getAuthToken(IntegrationService.java:67)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.manager.kickstart.cobbler.CobblerCommand.<init>(CobblerCommand.java:68)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.manager.kickstart.cobbler.CobblerDistroSyncCommand.<init>(CobblerDistroSyncCommand.java:52)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.taskomatic.task.CobblerSyncTask.execute(CobblerSyncTask.java:78)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.taskomatic.task.RhnJavaJob.execute(RhnJavaJob.java:88)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.taskomatic.TaskoJob.execute(TaskoJob.java:186)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
INFO   | jvm 1    | 2018/06/20 17:12:00 | Caused by: redstone.xmlrpc.XmlRpcFault: <class 'cobbler.cexceptions.CX'>:'login failed (taskomatic_user)'
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at redstone.xmlrpc.XmlRpcClient.handleResponse(XmlRpcClient.java:443)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at redstone.xmlrpc.XmlRpcClient.endCall(XmlRpcClient.java:376)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at redstone.xmlrpc.XmlRpcClient.invoke(XmlRpcClient.java:165)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.manager.kickstart.cobbler.CobblerXMLRPCHelper.invokeMethod(CobblerXMLRPCHelper.java:69)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               at com.redhat.rhn.manager.kickstart.cobbler.CobblerLoginCommand.login(CobblerLoginCommand.java:52)
INFO   | jvm 1    | 2018/06/20 17:12:00 |               ... 9 more
INFO   | jvm 1    | 2018/06/20 17:12:00 |

Rhn_taskomatic.log
[2018-06-20 15:12:00,046] INFO  com.redhat.rhn.taskomatic.TaskoJob - errata-queue-default: bunch errata-queue-bunch STARTED
[2018-06-20 15:12:00,046] DEBUG com.redhat.rhn.taskomatic.TaskoJob - errata-queue-default: task errata-queue started
[2018-06-20 15:12:00,079] INFO  com.redhat.rhn.taskomatic.TaskoJob - cobbler-sync-default: bunch cobbler-sync-bunch STARTED
[2018-06-20 15:12:00,079] DEBUG com.redhat.rhn.taskomatic.TaskoJob - cobbler-sync-default: task cobbler-sync started
[2018-06-20 15:12:00,153] ERROR com.redhat.rhn.taskomatic.task.CobblerSyncTask - RuntimeExceptionError trying to sync to cobbler: We had an error trying to login.
com.redhat.rhn.manager.kickstart.cobbler.NoCobblerTokenException: We had an error trying to login.
                at com.redhat.rhn.manager.kickstart.cobbler.CobblerLoginCommand.login(CobblerLoginCommand.java:57)
                at com.redhat.rhn.frontend.integration.IntegrationService.authorize(IntegrationService.java:114)
                at com.redhat.rhn.frontend.integration.IntegrationService.getAuthToken(IntegrationService.java:67)
                at com.redhat.rhn.manager.kickstart.cobbler.CobblerCommand.<init>(CobblerCommand.java:68)
                at com.redhat.rhn.manager.kickstart.cobbler.CobblerDistroSyncCommand.<init>(CobblerDistroSyncCommand.java:52)
                at com.redhat.rhn.taskomatic.task.CobblerSyncTask.execute(CobblerSyncTask.java:78)
                at com.redhat.rhn.taskomatic.task.RhnJavaJob.execute(RhnJavaJob.java:88)
                at com.redhat.rhn.taskomatic.TaskoJob.execute(TaskoJob.java:186)
                at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
                at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
Caused by: redstone.xmlrpc.XmlRpcFault: <class 'cobbler.cexceptions.CX'>:'login failed (taskomatic_user)'
                at redstone.xmlrpc.XmlRpcClient.handleResponse(XmlRpcClient.java:443)
                at redstone.xmlrpc.XmlRpcClient.endCall(XmlRpcClient.java:376)
                at redstone.xmlrpc.XmlRpcClient.invoke(XmlRpcClient.java:165)
                at com.redhat.rhn.manager.kickstart.cobbler.CobblerXMLRPCHelper.invokeMethod(CobblerXMLRPCHelper.java:69)
                at com.redhat.rhn.manager.kickstart.cobbler.CobblerLoginCommand.login(CobblerLoginCommand.java:52)
                ... 9 more
[2018-06-20 15:12:00,153] ERROR com.redhat.rhn.taskomatic.task.CobblerSyncTask - re-throwing exception since we havent yet.
[2018-06-20 15:12:00,154] ERROR com.redhat.rhn.taskomatic.task.CobblerSyncTask - Executing a task threw an exception: com.redhat.rhn.manager.kickstart.cobbler.NoCobblerTokenException
[2018-06-20 15:12:00,154] ERROR com.redhat.rhn.taskomatic.task.CobblerSyncTask - Message: We had an error trying to login.
[2018-06-20 15:12:00,154] ERROR com.redhat.rhn.taskomatic.task.CobblerSyncTask - Cause: redstone.xmlrpc.XmlRpcFault: <class 'cobbler.cexceptions.CX'>:'login failed (taskomatic_user)'
[2018-06-20 15:12:00,154] ERROR com.redhat.rhn.taskomatic.task.CobblerSyncTask - Stack trace:com.redhat.rhn.manager.kickstart.cobbler.NoCobblerTokenException: We had an error trying to login.
                at com.redhat.rhn.manager.kickstart.cobbler.CobblerLoginCommand.login(CobblerLoginCommand.java:57)
                at com.redhat.rhn.frontend.integration.IntegrationService.authorize(IntegrationService.java:114)
                at com.redhat.rhn.frontend.integration.IntegrationService.getAuthToken(IntegrationService.java:67)
                at com.redhat.rhn.manager.kickstart.cobbler.CobblerCommand.<init>(CobblerCommand.java:68)
                at com.redhat.rhn.manager.kickstart.cobbler.CobblerDistroSyncCommand.<init>(CobblerDistroSyncCommand.java:52)
                at com.redhat.rhn.taskomatic.task.CobblerSyncTask.execute(CobblerSyncTask.java:78)
                at com.redhat.rhn.taskomatic.task.RhnJavaJob.execute(RhnJavaJob.java:88)
                at com.redhat.rhn.taskomatic.TaskoJob.execute(TaskoJob.java:186)
                at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
                at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
Caused by: redstone.xmlrpc.XmlRpcFault: <class 'cobbler.cexceptions.CX'>:'login failed (taskomatic_user)'
                at redstone.xmlrpc.XmlRpcClient.handleResponse(XmlRpcClient.java:443)
                at redstone.xmlrpc.XmlRpcClient.endCall(XmlRpcClient.java:376)
                at redstone.xmlrpc.XmlRpcClient.invoke(XmlRpcClient.java:165)
                at com.redhat.rhn.manager.kickstart.cobbler.CobblerXMLRPCHelper.invokeMethod(CobblerXMLRPCHelper.java:69)
                at com.redhat.rhn.manager.kickstart.cobbler.CobblerLoginCommand.login(CobblerLoginCommand.java:52)
                ... 9 more

[2018-06-20 15:12:00,157] INFO  com.redhat.rhn.taskomatic.TaskoJob - channel-repodata-default: bunch channel-repodata-bunch STARTED
[2018-06-20 15:12:00,157] DEBUG com.redhat.rhn.taskomatic.TaskoJob - channel-repodata-default: task channel-repodata started
[2018-06-20 15:12:00,199] DEBUG com.redhat.rhn.taskomatic.TaskoJob - cobbler-sync (cobbler-sync-default) ... failed
[2018-06-20 15:12:00,199] INFO  com.redhat.rhn.taskomatic.TaskoJob - cobbler-sync-default: bunch cobbler-sync-bunch FINISHED
[2018-06-20 15:12:00,280] INFO  com.redhat.rhn.taskomatic.TaskoJob - errata-cache-default: bunch errata-cache-bunch STARTED
[2018-06-20 15:12:00,280] DEBUG com.redhat.rhn.taskomatic.TaskoJob - errata-cache-default: task errata-cache started
[2018-06-20 15:12:00,288] DEBUG com.redhat.rhn.taskomatic.TaskoJob - channel-repodata (channel-repodata-default) ... finished
[2018-06-20 15:12:00,288] INFO  com.redhat.rhn.taskomatic.TaskoJob - channel-repodata-default: bunch channel-repodata-bunch FINISHED
[2018-06-20 15:12:00,332] DEBUG com.redhat.rhn.taskomatic.TaskoJob - errata-queue (errata-queue-default) ... ready
[2018-06-20 15:12:00,332] INFO  com.redhat.rhn.taskomatic.TaskoJob - errata-queue-default: bunch errata-queue-bunch FINISHED
[2018-06-20 15:12:00,343] DEBUG com.redhat.rhn.taskomatic.TaskoJob - errata-cache (errata-cache-default) ... finished
[2018-06-20 15:12:00,343] DEBUG com.redhat.rhn.taskomatic.TaskoJob - errata-cache-default: task errata-mailer started
[2018-06-20 15:12:00,587] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Found 8 user(s) to notify about erratum 23692 in channel 142 for org 3.
[2018-06-20 15:12:00,588] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user 670admin(3) about 1 relevant server(s).
[2018-06-20 15:12:00,671] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user mgreason670(7) about 1 relevant server(s).
[2018-06-20 15:12:00,737] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user jwbradfo670(8) about 1 relevant server(s).
[2018-06-20 15:12:00,804] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user gcanter670(9) about 1 relevant server(s).
[2018-06-20 15:12:00,859] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user druvolo670(10) about 1 relevant server(s).
[2018-06-20 15:12:00,926] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user vhoang670(11) about 1 relevant server(s).
[2018-06-20 15:12:00,982] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user nbrizuel670(12) about 1 relevant server(s).
[2018-06-20 15:12:01,048] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user rnadjma670(13) about 1 relevant server(s).
[2018-06-20 15:12:01,154] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Found 8 user(s) to notify about erratum 23692 in channel 142 for org 2.
[2018-06-20 15:12:01,155] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user marida(16) about 40 relevant server(s).
[2018-06-20 15:12:01,272] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user aeclark1(32) about 40 relevant server(s).
[2018-06-20 15:12:01,338] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user scblumbe(33) about 40 relevant server(s).
[2018-06-20 15:12:01,404] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user 660admin(2) about 40 relevant server(s).
[2018-06-20 15:12:01,460] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user dpereira(34) about 40 relevant server(s).
[2018-06-20 15:12:01,515] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user yjpark(35) about 40 relevant server(s).
[2018-06-20 15:12:01,571] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user mgreason660(6) about 40 relevant server(s).
[2018-06-20 15:12:01,627] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user sfantasi660(25) about 40 relevant server(s).
[2018-06-20 15:12:01,873] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Found 8 user(s) to notify about erratum 23683 in channel 145 for org 2.
[2018-06-20 15:12:01,873] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user aeclark1(32) about 11 relevant server(s).
[2018-06-20 15:12:01,938] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user marida(16) about 11 relevant server(s).
[2018-06-20 15:12:01,994] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user scblumbe(33) about 11 relevant server(s).
[2018-06-20 15:12:02,060] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user dpereira(34) about 11 relevant server(s).
[2018-06-20 15:12:02,116] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user 660admin(2) about 11 relevant server(s).
[2018-06-20 15:12:02,172] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user yjpark(35) about 11 relevant server(s).
[2018-06-20 15:12:02,227] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user mgreason660(6) about 11 relevant server(s).
[2018-06-20 15:12:02,283] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user sfantasi660(25) about 11 relevant server(s).
[2018-06-20 15:12:02,418] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Found 8 user(s) to notify about erratum 23683 in channel 164 for org 3.
[2018-06-20 15:12:02,419] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user 670admin(3) about 1 relevant server(s).
[2018-06-20 15:12:02,483] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user mgreason670(7) about 1 relevant server(s).
[2018-06-20 15:12:02,550] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user jwbradfo670(8) about 1 relevant server(s).
[2018-06-20 15:12:02,617] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user gcanter670(9) about 1 relevant server(s).
[2018-06-20 15:12:02,683] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user druvolo670(10) about 1 relevant server(s).
[2018-06-20 15:12:02,750] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user vhoang670(11) about 1 relevant server(s).
[2018-06-20 15:12:02,817] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user nbrizuel670(12) about 1 relevant server(s).
[2018-06-20 15:12:02,884] INFO  com.redhat.rhn.taskomatic.task.ErrataMailer - Notification for user rnadjma670(13) about 1 relevant server(s).
[2018-06-20 15:12:02,999] DEBUG com.redhat.rhn.taskomatic.TaskoJob - errata-mailer (errata-cache-default) ... finished
[2018-06-20 15:12:02,999] INFO  com.redhat.rhn.taskomatic.TaskoJob - errata-cache-default: bunch errata-cache-bunch FINISHED

Kickstart profile log
Jun 20, 2018 4:52:36 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [action] in context with path [/rhn] threw exception [com.redhat.rhn.manager.kickstart.cobbler.NoCobblerTokenException: We had an error trying to login.] with root cause
redstone.xmlrpc.XmlRpcFault: <class 'cobbler.cexceptions.CX'>:'login failed (admin)'
                at redstone.xmlrpc.XmlRpcClient.handleResponse(XmlRpcClient.java:443)
                at redstone.xmlrpc.XmlRpcClient.endCall(XmlRpcClient.java:376)
                at redstone.xmlrpc.XmlRpcClient.invoke(XmlRpcClient.java:165)
                at com.redhat.rhn.manager.kickstart.cobbler.CobblerXMLRPCHelper.invokeMethod(CobblerXMLRPCHelper.java:69)
                at com.redhat.rhn.manager.kickstart.cobbler.CobblerLoginCommand.login(CobblerLoginCommand.java:52)
                at com.redhat.rhn.frontend.integration.IntegrationService.authorize(IntegrationService.java:114)
                at com.redhat.rhn.frontend.integration.IntegrationService.getAuthToken(IntegrationService.java:67)
                at com.redhat.rhn.manager.kickstart.cobbler.CobblerXMLRPCHelper.getConnection(CobblerXMLRPCHelper.java:92)
                at com.redhat.rhn.manager.kickstart.cobbler.CobblerXMLRPCHelper.getConnection(CobblerXMLRPCHelper.java:83)
                at com.redhat.rhn.manager.kickstart.KickstartLister.listCobblerProfiles(KickstartLister.java:438)
                at com.redhat.rhn.frontend.action.kickstart.KickstartsSetupAction.execute(KickstartsSetupAction.java:63)
                at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:425)
                at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:229)
                at com.redhat.rhn.frontend.struts.RhnRequestProcessor.process(RhnRequestProcessor.java:105)
                at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1913)
                at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:449)
                at javax.servlet.http.HttpServlet.service(HttpServlet.java:624)
                at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
                at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
                at com.redhat.rhn.frontend.servlets.AuthFilter.doFilter(AuthFilter.java:107)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
                at com.opensymphony.sitemesh.webapp.SiteMeshFilter.obtainContent(SiteMeshFilter.java:129)
                at com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:77)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
                at com.redhat.rhn.frontend.servlets.LocalizedEnvironmentFilter.doFilter(LocalizedEnvironmentFilter.java:67)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
                at com.redhat.rhn.frontend.servlets.EnvironmentFilter.doFilter(EnvironmentFilter.java:101)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
                at com.redhat.rhn.frontend.servlets.SessionFilter.doFilter(SessionFilter.java:57)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
                at com.redhat.rhn.frontend.servlets.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:97)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
                at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
                at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
                at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506)
                at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
                at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
                at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
                at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
                at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
                at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:190)
                at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
                at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
                at java.lang.Thread.run(Thread.java:748)

Jun 20, 2018 4:54:25 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [action] in context with path [/rhn] threw exception [com.redhat.rhn.manager.kickstart.cobbler.NoCobblerTokenException: We had an error trying to login.] with root cause
redstone.xmlrpc.XmlRpcFault: <class 'cobbler.cexceptions.CX'>:'login failed (admin)'
                at redstone.xmlrpc.XmlRpcClient.handleResponse(XmlRpcClient.java:443)
                at redstone.xmlrpc.XmlRpcClient.endCall(XmlRpcClient.java:376)
                at redstone.xmlrpc.XmlRpcClient.invoke(XmlRpcClient.java:165)
                at com.redhat.rhn.manager.kickstart.cobbler.CobblerXMLRPCHelper.invokeMethod(CobblerXMLRPCHelper.java:69)
                at com.redhat.rhn.manager.kickstart.cobbler.CobblerLoginCommand.login(CobblerLoginCommand.java:52)
                at com.redhat.rhn.frontend.integration.IntegrationService.authorize(IntegrationService.java:114)
                at com.redhat.rhn.frontend.integration.IntegrationService.getAuthToken(IntegrationService.java:67)
                at com.redhat.rhn.manager.kickstart.cobbler.CobblerXMLRPCHelper.getConnection(CobblerXMLRPCHelper.java:92)
                at com.redhat.rhn.manager.kickstart.cobbler.CobblerXMLRPCHelper.getConnection(CobblerXMLRPCHelper.java:83)
                at com.redhat.rhn.manager.kickstart.KickstartLister.listCobblerProfiles(KickstartLister.java:438)
                at com.redhat.rhn.frontend.action.kickstart.KickstartsSetupAction.execute(KickstartsSetupAction.java:63)
                at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:425)
                at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:229)
                at com.redhat.rhn.frontend.struts.RhnRequestProcessor.process(RhnRequestProcessor.java:105)
                at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1913)
                at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:449)
                at javax.servlet.http.HttpServlet.service(HttpServlet.java:624)
                at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
                at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
                at com.redhat.rhn.frontend.servlets.AuthFilter.doFilter(AuthFilter.java:107)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
                at com.opensymphony.sitemesh.webapp.SiteMeshFilter.obtainContent(SiteMeshFilter.java:129)
                at com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:77)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
                at com.redhat.rhn.frontend.servlets.LocalizedEnvironmentFilter.doFilter(LocalizedEnvironmentFilter.java:67)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
                at com.redhat.rhn.frontend.servlets.EnvironmentFilter.doFilter(EnvironmentFilter.java:101)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
                at com.redhat.rhn.frontend.servlets.SessionFilter.doFilter(SessionFilter.java:57)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
                at com.redhat.rhn.frontend.servlets.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:97)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
                at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
                at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
                at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506)
                at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
                at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
                at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
                at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
                at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
                at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:190)
                at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
                at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
                at java.lang.Thread.run(Thread.java:748)

Note: I don’t see any activity in the Apache logs corresponding to these requests.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/spacewalk-list/attachments/20180620/7352403b/attachment.htm>


More information about the Spacewalk-list mailing list