What version of PostgreSQL are you running?<br><br><span style="font-family:Prelude, Verdana, san-serif;"><br><br></span><span id="signature"><div style="font-family: arial, sans-serif; font-size: 12px;color: #999999;">-- Sent from my HP Pre3</div><br></span><span style="color:navy; font-family:Prelude, Verdana, san-serif; "><hr align="left" style="width:75%">On Dec 23, 2014 5:20 PM, Jagga Soorma <jagga13@gmail.com> wrote: <br><br></span>Since I am not using this in production yet, I decided to upgrade <br>everything to the latest available on the spacewalk server running <br>CentOS.  Did not touch the external postgres server running on RHEL. <br>However, still getting the same issue.  This is the complete error <br>when in the rhn_taskomatic_daemon.log file when restarting spacewalk: <br> <br>-- <br>STATUS | wrapper  | 2014/12/23 14:09:31 | TERM trapped.  Shutting down. <br>STATUS | wrapper  | 2014/12/23 14:09:32 | <-- Wrapper Stopped <br>STATUS | wrapper  | 2014/12/23 14:10:08 | --> Wrapper Started as Daemon <br>STATUS | wrapper  | 2014/12/23 14:10:08 | Launching a JVM... <br>INFO   | jvm 1    | 2014/12/23 14:10:08 | Wrapper (Version 3.2.3) <br>http://wrapper.tanukisoftware.org <br>INFO   | jvm 1    | 2014/12/23 14:10:08 |   Copyright 1999-2006 Tanuki <br>Software, Inc.  All Rights Reserved. <br>INFO   | jvm 1    | 2014/12/23 14:10:08 | <br>INFO   | jvm 1    | 2014/12/23 14:10:12 | Dec 23, 2014 2:10:12 PM <br>com.mchange.v2.log.MLog <clinit> <br>INFO   | jvm 1    | 2014/12/23 14:10:12 | INFO: MLog clients using <br>java 1.4+ standard logging. <br>INFO   | jvm 1    | 2014/12/23 14:10:12 | Dec 23, 2014 2:10:12 PM <br>com.mchange.v2.c3p0.C3P0Registry banner <br>INFO   | jvm 1    | 2014/12/23 14:10:12 | INFO: Initializing <br>c3p0-0.9.1.2 [built 06-August-2008 15:35:00; debug? false; trace: 5] <br>INFO   | jvm 1    | 2014/12/23 14:10:12 | Dec 23, 2014 2:10:12 PM <br>com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource getPoolManager <br>INFO   | jvm 1    | 2014/12/23 14:10:12 | INFO: Initializing c3p0 <br>pool... com.mchange.v2.c3p0.PoolBackedDataSource@585a1c43 [ <br>connectionPoolDataSource -> <br>com.mchange.v2.c3p0.WrapperConnectionPoolDataSource@1c136579 [ <br>acquireIncrement -> 3, acquireRetryAttempts -> 30, acquireRetryDelay <br>-> 1000, autoCommitOnClose -> false, automaticTestTable -> null, <br>breakAfterAcquireFailure -> false, checkoutTimeout -> 0, <br>connectionCustomizerClassName -> <br>com.redhat.rhn.common.db.RhnConnectionCustomizer, <br>connectionTesterClassName -> <br>com.mchange.v2.c3p0.impl.DefaultConnectionTester, <br>debugUnreturnedConnectionStackTraces -> false, factoryClassLocation -> <br>null, forceIgnoreUnresolvedTransactions -> false, identityToken -> <br>k0jcdo96xmmxcv1leoshp|2d09bbf2, idleConnectionTestPeriod -> 300, <br>initialPoolSize -> 5, maxAdministrativeTaskTime -> 0, maxConnectionAge <br>-> 0, maxIdleTime -> 300, maxIdleTimeExcessConnections -> 0, <br>maxPoolSize -> 20, maxStatements -> 0, maxStatementsPerConnection -> <br>0, minPoolSize -> 5, nestedDataSource -> <br>com.mchange.v2.c3p0.DriverManagerDataSource@f7c08283 [ description -> <br>null, driverClass -> null, factoryClassLocation -> null, identityToken <br>-> k0jcdo96xmmxcv1leoshp|7451e191, jdbcUrl -> <br>jdbc:postgresql://rwcswalkpg01:5432/spaceschema, properties -> <br>{user=******, password=******, driver_proto=jdbc:postgresql} ], <br>preferredTestQuery -> select 'c3p0 ping' from dual, propertyCycle -> <br>0, testConnectionOnCheckin -> false, testConnectionOnCheckout -> true, <br>unreturnedConnectionTimeout -> 0, usesTraditionalReflectiveProxies -> <br>false; userOverrides: {} ], dataSourceName -> null, <br>factoryClassLocation -> null, identityToken -> <br>k0jcdo96xmmxcv1leoshp|2fe19787, numHelperThreads -> 3 ] <br>FATAL  | jvm 1    | 2014/12/23 14:10:16 | Failure occured during job recovery. <br>com.redhat.rhn.taskomatic.core.TaskomaticException: Failure occured <br>during job recovery. <br>at com.redhat.rhn.taskomatic.core.SchedulerKernel.startup(SchedulerKernel.java:158) <br>at com.redhat.rhn.taskomatic.core.TaskomaticDaemon$1.run(TaskomaticDaemon.java:102) <br>at java.lang.Thread.run(Thread.java:745) <br>Caused by: org.quartz.SchedulerConfigException: Failure occured during <br>job recovery. [See nested exception: <br>org.quartz.JobPersistenceException: Couldn't retrieve trigger: 2 [See <br>nested exception: java.lang.ArrayIndexOutOfBoundsException: 2]] <br>at org.quartz.impl.jdbcjobstore.JobStoreSupport.schedulerStarted(JobStoreSupport.java:627) <br>at org.quartz.core.QuartzScheduler.start(QuartzScheduler.java:494) <br>at org.quartz.impl.StdScheduler.start(StdScheduler.java:143) <br>at com.redhat.rhn.taskomatic.core.SchedulerKernel.startup(SchedulerKernel.java:146) <br>... 2 more <br>Caused by: org.quartz.JobPersistenceException: Couldn't retrieve <br>trigger: 2 [See nested exception: <br>java.lang.ArrayIndexOutOfBoundsException: 2] <br>at org.quartz.impl.jdbcjobstore.JobStoreSupport.retrieveTrigger(JobStoreSupport.java:1571) <br>at org.quartz.impl.jdbcjobstore.JobStoreSupport.recoverMisfiredJobs(JobStoreSupport.java:950) <br>at org.quartz.impl.jdbcjobstore.JobStoreSupport.recoverJobs(JobStoreSupport.java:834) <br>at org.quartz.impl.jdbcjobstore.JobStoreSupport$2.execute(JobStoreSupport.java:806) <br>at org.quartz.impl.jdbcjobstore.JobStoreSupport$41.execute(JobStoreSupport.java:3729) <br>at org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3763) <br>at org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3725) <br>at org.quartz.impl.jdbcjobstore.JobStoreSupport.recoverJobs(JobStoreSupport.java:802) <br>at org.quartz.impl.jdbcjobstore.JobStoreSupport.schedulerStarted(JobStoreSupport.java:625) <br>... 5 more <br>Caused by: java.lang.ArrayIndexOutOfBoundsException: 2 <br>at org.postgresql.util.PGbytea.toBytes(PGbytea.java:74) <br>at org.postgresql.jdbc2.AbstractJdbc2ResultSet.getBytes(AbstractJdbc2ResultSet.java:2253) <br>at org.postgresql.jdbc2.AbstractJdbc2ResultSet.getBytes(AbstractJdbc2ResultSet.java:2433) <br>at org.apache.commons.dbcp.DelegatingResultSet.getBytes(DelegatingResultSet.java:253) <br>at org.quartz.impl.jdbcjobstore.PostgreSQLDelegate.getObjectFromBlob(PostgreSQLDelegate.java:92) <br>at org.quartz.impl.jdbcjobstore.StdJDBCDelegate.selectTrigger(StdJDBCDelegate.java:2132) <br>at org.quartz.impl.jdbcjobstore.JobStoreSupport.retrieveTrigger(JobStoreSupport.java:1553) <br>... 13 more <br>-- <br> <br>Need help with troubleshooting this because I can't seem to get any further. <br> <br>Thanks! <br> <br>On Tue, Dec 23, 2014 at 1:25 PM, Jagga Soorma <jagga13@gmail.com> wrote: <br>> Hi Guys, <br>> <br>> I am setting up a new satellite server with a external postgres db and just <br>> cloned my first CentOS base/extras channel.  On the client side everything <br>> seems to have gone smoothly except when I now run "yum repolist" I see 0 <br>> packages for these 2 channels even though there are packages in those <br>> channels.  One thing I have noticed is that I am seeing tons of the <br>> following errors in my /var/log/rhn/rhn_taskomatic_daemon.log log file on <br>> the spacewalk server: <br>> <br>> -- <br>> ..snip.. <br>> INFO   | jvm 1    | 2014/12/23 13:16:23 | 2014-12-23 13:16:23,458 <br>> [QuartzScheduler_DefaultQuartzScheduler-NON_CLUSTERED_MisfireHandler] ERROR <br>> org.quartz.impl.jdbcjobstore.JobStoreTX - MisfireHandler: Error handling <br>> misfires: Couldn't retrieve trigger: 2 <br>> INFO   | jvm 1    | 2014/12/23 13:16:23 | <br>> org.quartz.JobPersistenceException: Couldn't retrieve trigger: 2 [See nested <br>> exception: java.lang.ArrayIndexOutOfBoundsException: 2] <br>> INFO   | jvm 1    | 2014/12/23 13:16:23 | at <br>> org.quartz.impl.jdbcjobstore.JobStoreSupport.retrieveTrigger(JobStoreSupport.java:1571) <br>> INFO   | jvm 1    | 2014/12/23 13:16:23 | at <br>> org.quartz.impl.jdbcjobstore.JobStoreSupport.recoverMisfiredJobs(JobStoreSupport.java:950) <br>> INFO   | jvm 1    | 2014/12/23 13:16:23 | at <br>> org.quartz.impl.jdbcjobstore.JobStoreSupport.doRecoverMisfires(JobStoreSupport.java:3125) <br>> INFO   | jvm 1    | 2014/12/23 13:16:23 | at <br>> org.quartz.impl.jdbcjobstore.JobStoreSupport$MisfireHandler.manage(JobStoreSupport.java:3896) <br>> INFO   | jvm 1    | 2014/12/23 13:16:23 | at <br>> org.quartz.impl.jdbcjobstore.JobStoreSupport$MisfireHandler.run(JobStoreSupport.java:3916) <br>> INFO   | jvm 1    | 2014/12/23 13:16:23 | Caused by: <br>> java.lang.ArrayIndexOutOfBoundsException: 2 <br>> INFO   | jvm 1    | 2014/12/23 13:16:23 | at <br>> org.postgresql.util.PGbytea.toBytes(PGbytea.java:76) <br>> INFO   | jvm 1    | 2014/12/23 13:16:23 | at <br>> org.postgresql.jdbc2.AbstractJdbc2ResultSet.getBytes(AbstractJdbc2ResultSet.java:2271) <br>> INFO   | jvm 1    | 2014/12/23 13:16:23 | at <br>> org.postgresql.jdbc2.AbstractJdbc2ResultSet.getBytes(AbstractJdbc2ResultSet.java:2451) <br>> INFO   | jvm 1    | 2014/12/23 13:16:23 | at <br>> org.apache.commons.dbcp.DelegatingResultSet.getBytes(DelegatingResultSet.java:253) <br>> INFO   | jvm 1    | 2014/12/23 13:16:23 | at <br>> org.quartz.impl.jdbcjobstore.PostgreSQLDelegate.getObjectFromBlob(PostgreSQLDelegate.java:92) <br>> INFO   | jvm 1    | 2014/12/23 13:16:23 | at <br>> org.quartz.impl.jdbcjobstore.StdJDBCDelegate.selectTrigger(StdJDBCDelegate.java:2132) <br>> INFO   | jvm 1    | 2014/12/23 13:16:23 | at <br>> org.quartz.impl.jdbcjobstore.JobStoreSupport.retrieveTrigger(JobStoreSupport.java:1553) <br>> INFO   | jvm 1    | 2014/12/23 13:16:23 | ... 4 more <br>> INFO   | jvm 1    | 2014/12/23 13:20:23 | 2014-12-23 13:20:23,458 <br>> [QuartzScheduler_DefaultQuartzScheduler-NON_CLUSTERED_MisfireHandler] ERROR <br>> org.quartz.impl.jdbcjobstore.JobStoreTX - MisfireHandler: Error handling <br>> misfires: Couldn't retrieve trigger: 2 <br>> INFO   | jvm 1    | 2014/12/23 13:20:23 | <br>> org.quartz.JobPersistenceException: Couldn't retrieve trigger: 2 [See nested <br>> exception: java.lang.ArrayIndexOutOfBoundsException: 2] <br>> INFO   | jvm 1    | 2014/12/23 13:20:23 | at <br>> org.quartz.impl.jdbcjobstore.JobStoreSupport.retrieveTrigger(JobStoreSupport.java:1571) <br>> INFO   | jvm 1    | 2014/12/23 13:20:23 | at <br>> org.quartz.impl.jdbcjobstore.JobStoreSupport.recoverMisfiredJobs(JobStoreSupport.java:950) <br>> INFO   | jvm 1    | 2014/12/23 13:20:23 | at <br>> org.quartz.impl.jdbcjobstore.JobStoreSupport.doRecoverMisfires(JobStoreSupport.java:3125) <br>> INFO   | jvm 1    | 2014/12/23 13:20:23 | at <br>> org.quartz.impl.jdbcjobstore.JobStoreSupport$MisfireHandler.manage(JobStoreSupport.java:3896) <br>> INFO   | jvm 1    | 2014/12/23 13:20:23 | at <br>> org.quartz.impl.jdbcjobstore.JobStoreSupport$MisfireHandler.run(JobStoreSupport.java:3916) <br>> INFO   | jvm 1    | 2014/12/23 13:20:23 | Caused by: <br>> java.lang.ArrayIndexOutOfBoundsException: 2 <br>> INFO   | jvm 1    | 2014/12/23 13:20:23 | at <br>> org.postgresql.util.PGbytea.toBytes(PGbytea.java:76) <br>> INFO   | jvm 1    | 2014/12/23 13:20:23 | at <br>> org.postgresql.jdbc2.AbstractJdbc2ResultSet.getBytes(AbstractJdbc2ResultSet.java:2271) <br>> INFO   | jvm 1    | 2014/12/23 13:20:23 | at <br>> org.postgresql.jdbc2.AbstractJdbc2ResultSet.getBytes(AbstractJdbc2ResultSet.java:2451) <br>> INFO   | jvm 1    | 2014/12/23 13:20:23 | at <br>> org.apache.commons.dbcp.DelegatingResultSet.getBytes(DelegatingResultSet.java:253) <br>> INFO   | jvm 1    | 2014/12/23 13:20:23 | at <br>> org.quartz.impl.jdbcjobstore.PostgreSQLDelegate.getObjectFromBlob(PostgreSQLDelegate.java:92) <br>> INFO   | jvm 1    | 2014/12/23 13:20:23 | at <br>> org.quartz.impl.jdbcjobstore.StdJDBCDelegate.selectTrigger(StdJDBCDelegate.java:2132) <br>> INFO   | jvm 1    | 2014/12/23 13:20:23 | at <br>> org.quartz.impl.jdbcjobstore.JobStoreSupport.retrieveTrigger(JobStoreSupport.java:1553) <br>> INFO   | jvm 1    | 2014/12/23 13:20:23 | ... 4 more <br>> ..snip.. <br>> -- <br>> <br>> Any help would be greatly appreciated. <br>> <br>> Thanks! <br> <br>_______________________________________________ <br>Spacewalk-list mailing list <br>Spacewalk-list@redhat.com <br>https://www.redhat.com/mailman/listinfo/spacewalk-list <br>