[Spacewalk-list] taskomatic/reposync schedules stop after 24h

Thu Oct 2 11:36:36 UTC 2014

Dear list,

I'm on spacewalk 2.0/Oracle 11g (not Express) and I have trouble with my 
channel synching. When I restart spacewalk, a full sync of all channels 
goes through, but despite having all channels configured for daily sync, 
they do no longer sync the second day.
I have this in /var/log/rhn/rhn_taskomatic_daemon.log:

INFO   | jvm 1    | 2014/10/01 22:04:25 | Oct 1, 2014 10:04:25 PM 
com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector run
INFO   | jvm 1    | 2014/10/01 22:04:25 | WARNING: 
com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector at 1364ee1f 
-- APPARENT DEADLOCK!!! Creating emergency threads
 for unassigned pending tasks!
INFO   | jvm 1    | 2014/10/01 22:04:25 | Oct 1, 2014 10:04:25 PM 
com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector run
INFO   | jvm 1    | 2014/10/01 22:04:25 | WARNING: 
com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector at 1364ee1f 
-- APPARENT DEADLOCK!!! Complete Status: 
INFO   | jvm 1    | 2014/10/01 22:04:25 |       Managed Threads: 3
INFO   | jvm 1    | 2014/10/01 22:04:25 |       Active Threads: 3
INFO   | jvm 1    | 2014/10/01 22:04:25 |       Active Tasks: 
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask at 44e6c939 
(com.mchange.v2.async.ThreadPoolAsynchronousR
unner$PoolThread-#2)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask at ba4e086 
(com.mchange.v2.async.ThreadPoolAsynchronousRu
nner$PoolThread-#0)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask at 75908362 
(com.mchange.v2.async.ThreadPoolAsynchronousR
unner$PoolThread-#1)
INFO   | jvm 1    | 2014/10/01 22:04:25 |       Pending Tasks: 
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask at 62a1d33a
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask at 6e32b76c
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask at 43a84bf9
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask at 5e13eb22
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask at 6db2356e
INFO   | jvm 1    | 2014/10/01 22:04:25 | Pool thread stack traces:
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2,5,main]
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
oracle.jdbc.driver.T2CConnection.t2cLogoff(Native Method)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
oracle.jdbc.driver.T2CConnection.logoff(T2CConnection.java:466)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
oracle.jdbc.driver.PhysicalConnection.close(PhysicalConnection.java:3984)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
oracle.jdbc.driver.T2CConnection.close(T2CConnection.java:47)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.c3p0.impl.NewPooledConnection.close(NewPooledConnection.java:549)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.c3p0.impl.NewPooledConnection.close(NewPooledConnection.java:234)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.destroyResource(C3P0PooledConnectionPool.java:470)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask.run(BasicResourcePool.java:964)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0,5,main]
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
oracle.jdbc.driver.T2CConnection.t2cLogoff(Native Method)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
oracle.jdbc.driver.T2CConnection.logoff(T2CConnection.java:466)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
oracle.jdbc.driver.PhysicalConnection.close(PhysicalConnection.java:3984)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
oracle.jdbc.driver.T2CConnection.close(T2CConnection.java:47)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.c3p0.impl.NewPooledConnection.close(NewPooledConnection.java:549)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.c3p0.impl.NewPooledConnection.close(NewPooledConnection.java:234)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.destroyResource(C3P0PooledConnectionPool.java:470)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask.run(BasicResourcePool.java:964)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1,5,main]
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
oracle.jdbc.driver.T2CConnection.t2cLogoff(Native Method)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
oracle.jdbc.driver.T2CConnection.logoff(T2CConnection.java:466)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
oracle.jdbc.driver.PhysicalConnection.close(PhysicalConnection.java:3984)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
oracle.jdbc.driver.T2CConnection.close(T2CConnection.java:47)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.c3p0.impl.NewPooledConnection.close(NewPooledConnection.java:549)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.c3p0.impl.NewPooledConnection.close(NewPooledConnection.java:234)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.destroyResource(C3P0PooledConnectionPool.java:470)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask.run(BasicResourcePool.java:964)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
INFO   | jvm 1    | 2014/10/01 22:04:25 | 
STATUS | wrapper  | 2014/10/02 13:21:57 | TERM trapped.  Shutting down.
STATUS | wrapper  | 2014/10/02 13:21:58 | <-- Wrapper Stopped
STATUS | wrapper  | 2014/10/02 13:23:10 | --> Wrapper Started as Daemon
STATUS | wrapper  | 2014/10/02 13:23:10 | Launching a JVM...
INFO   | jvm 1    | 2014/10/02 13:23:11 | Wrapper (Version 3.2.3) 
http://wrapper.tanukisoftware.org
INFO   | jvm 1    | 2014/10/02 13:23:11 |   Copyright 1999-2006 Tanuki 
Software, Inc.  All Rights Reserved.
INFO   | jvm 1    | 2014/10/02 13:23:11 | 
INFO   | jvm 1    | 2014/10/02 13:23:22 | Oct 2, 2014 1:23:22 PM 
com.mchange.v2.log.MLog <clinit>
INFO   | jvm 1    | 2014/10/02 13:23:22 | INFO: MLog clients using java 
1.4+ standard logging.
INFO   | jvm 1    | 2014/10/02 13:23:22 | Oct 2, 2014 1:23:22 PM 
com.mchange.v2.c3p0.C3P0Registry banner
INFO   | jvm 1    | 2014/10/02 13:23:22 | INFO: Initializing c3p0-0.9.1.2 
[built 06-August-2008 15:35:00; debug? false; trace: 5]
INFO   | jvm 1    | 2014/10/02 13:23:22 | Oct 2, 2014 1:23:22 PM 
com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource getPoolManager
INFO   | jvm 1    | 2014/10/02 13:23:22 | INFO: Initializing c3p0 pool... 
com.mchange.v2.c3p0.PoolBackedDataSource at 6a0d0a4a [ 
connectionPoolDataSource -> 
com.mchange.v2.c3p0.WrapperConnectionPoolDataSource at f0c52e9f [ 
acquireIncrement -> 3, acquireRetryAttempts -> 30, acquireRetryDelay -> 
1000, autoCommitOnClose -> false, automaticTestTable -> null, 
breakAfterAcquireFailure -> false, checkoutTimeout -> 0, 
connectionCustomizerClassName -> 
com.redhat.rhn.common.db.RhnConnectionCustomizer, 
connectionTesterClassName -> 
com.mchange.v2.c3p0.impl.DefaultConnectionTester, 
debugUnreturnedConnectionStackTraces -> false, factoryClassLocation -> 
null, forceIgnoreUnresolvedTransactions -> false, identityToken -> 
2rvzfg941lvnch3155735j|e5a0fc1, idleConnectionTestPeriod -> 300, 
initialPoolSize -> 5, maxAdministrativeTaskTime -> 0, maxConnectionAge -> 
0, maxIdleTime -> 300, maxIdleTimeExcessConnections -> 0, maxPoolSize -> 
20, maxStatements -> 0, maxStatementsPerConnection -> 0, minPoolSize -> 5, 
nestedDataSource -> com.mchange.v2.c3p0.DriverManagerDataSource at 57195be0 [ 
description -> null, driverClass -> null, factoryClassLocation -> null, 
identityToken -> 2rvzfg941lvnch3155735j|6614418b, jdbcUrl -> 
jdbc:oracle:oci:@//mgmt@<xxxxxxxxx>, properties -> {user=******, 
password=******, driver_proto=jdbc:oracle:oci} ], preferredTestQuery -> 
select 'c3p0 ping' from dual, propertyCycle -> 0, testConnectionOnCheckin 
-> false, testConnectionOnCheckout -> true, unreturnedConnectionTimeout -> 
0, usesTraditionalReflectiveProxies -> false; userOverrides: {} ], 
dataSourceName -> null, factoryClassLocation -> null, identityToken -> 
2rvzfg941lvnch3155735j|6f277aab, numHelperThreads -> 3 ]
INFO   | jvm 1    | 2014/10/02 13:23:33 | 2014-10-02 13:23:33,423 
[Thread-43] WARN  com.redhat.rhn.taskomatic.core.SchedulerKernel - Number 
of interrupted runs: 1
INFO   | jvm 1    | 2014/10/02 13:23:45 | 2014-10-02 13:23:45,396 
[DefaultQuartzScheduler_Worker-2] INFO 
com.redhat.rhn.taskomatic.task.RepoSyncTask - Syncing repos for channel: 
CentOS 5 (x86_64) HEAD

the restart was at ~2014/10/02 13:22:00, after that the normal sync 
starts. Any suggestions what trouble I am looking at?

best regards
Jens Neu

www.biotronik.com 
BIOTRONIK - excellence for life 
Established with the development of the first German pacemaker in 1963, BIOTRONIK has upheld the highest quality standards in the fields of cardiac rhythm management and vascular intervention in more than 100 countries worldwide. We’ve developed advanced technologies and products such as BIOTRONIK Home Monitoring®, Closed Loop Stimulation (CLS) and Orsiro, the industry’s first hybrid drug eluting stent. BIOTRONIK also offers the broadest portfolio of cardiac devices with ProMRI®, an advanced technology that gives patients access to magnetic resonance (MR) scanning. 
BIOTRONIK SE & Co. KG 
Woermannkehre 1, 12359 Berlin, Germany 
Sitz der Gesellschaft: Berlin, Registergericht: Berlin HRA 6501 

Vertreten durch ihre Komplementärin: 
BIOTRONIK MT SE 
Sitz der Gesellschaft: Berlin, Registergericht: Berlin HRB 118866 B 
Geschäftsführende Direktoren: Christoph Böhmer, Dr. Lothar Krings 
This e-mail and the information it contains including attachments are confidential and meant only for use by the intended recipient(s); disclosure or copying is strictly prohibited. If you are not addressed, but in the possession of this e-mail, please notify the sender immediately and delete the document. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/spacewalk-list/attachments/20141002/9edd781d/attachment.htm>