[Freeipa-users] pki-tomcat failed. [SOLVED]
Bob Hinton
bob at rha-ltd.co.uk
Tue Jan 10 23:39:39 UTC 2017
Hi Adam,
With the change to ldap instead of ldaps on the CA master that you
suggested I was able to move the system clock to before the certificate
expiry time then do
ipactl start --ignore-service-failures
systemctl start pki-tomcat at pki-tomcat.service
then start the pki ca service manually as stated in "systemctl status
pki-tomcat at pki-tomcat.service -l"
service certmonger restart
I then ran "getcert list | more" and waited until all the certificates
had updated
After that "ipactl stop; ipactl start" completed without errors and I
could reenable ntpd and vmware tools timesync.
Finally ipa-certupdate seems to have been needed to propagate the new
certs to the other replicas.
Many thanks
Bob
On 10/01/2017 20:47, Adam Tkac wrote:
> Hello,
>
> we hit similar issue (although due to different conditions - we rotated
> root CA cert and then newly issued certificates were wrongly signed), we
> were also unable to start tomcat. If I remember correctly, we switched dogtag
> to use simple binds instead of TLS to connect to LDAP this way.
>
> 1. systemctl stop pki-tomcatd at pki-tomcat.service
> 2. backup /etc/pki/pki-tomcat/ca/CS.cfg and /etc/pki/pki-tomcat/password.conf
> 3. add directory manager password into password.conf file, it should be line
> like
>
> internaldb=my_directory_manager_password
>
> 4. tune CS.cfg a little, take this diff as an example
>
> +internaldb.ldapauth.authtype=BasicAuth
> +internaldb.ldapauth.bindDN=cn=Directory Manager
> +internaldb.ldapauth.bindPWPrompt=internaldb
> -internaldb.ldapauth.authtype=SslClientAuth
> -internaldb.ldapauth.bindDN=uid=pkidbuser,ou=people,o=ipaca
> internaldb.ldapauth.clientCertNickname=subsystemCert cert-pki-ca
> internaldb.ldapconn.cloneReplicationPort=389
> internaldb.ldapconn.masterReplicationPort=7389
> +internaldb.ldapconn.port=389
> -internaldb.ldapconn.port=636
> internaldb.ldapconn.replicationSecurity=TLS
> +internaldb.ldapconn.secureConn=false
> -internaldb.ldapconn.secureConn=true
>
> 5. systemctl start pki-tomcatd at pki-tomcat.service
>
> Now tomcat should run correctly and you should be able to resubmit expired
> certs and you can start to experiment with switch dogtag back to TLS auth.
> Hope this helps you.
>
> Regards, Adam
>
> On Tue, Jan 10, 2017 at 07:27:04PM +0000, Bob Hinton wrote:
>> Hi,
>>
>> The pki-tomcatd services on our IPA servers seem to have stopped working.
>>
>> This seems to be related to the expiry of several certificates -
>>
>> [root at ipa001 ~]# getcert list | more
>> Number of certificates and requests being tracked: 8.
>> Request ID '20161230150048':
>> status: MONITORING
>> stuck: no
>> key pair storage:
>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
>> cert-pki-ca',token='NSS Certificate DB',pin set
>> certificate:
>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
>> cert-pki-ca',token='NSS Certificate DB'
>> CA: dogtag-ipa-ca-renew-agent
>> issuer: CN=Certificate Authority,O=LOCAL.COM
>> subject: CN=CA Audit,O=LOCAL.COM
>> expires: 2017-01-09 08:21:45 UTC
>> key usage: digitalSignature,nonRepudiation
>> pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>> post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
>> "auditSigningCert cert-pki-ca"
>> track: yes
>> auto-renew: yes
>> Request ID '20161230150049':
>> status: MONITORING
>> stuck: no
>> key pair storage:
>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
>> cert-pki-ca',token='NSS Certificate DB',pin set
>> certificate:
>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
>> cert-pki-ca',token='NSS Certificate DB'
>> CA: dogtag-ipa-ca-renew-agent
>> issuer: CN=Certificate Authority,O=LOCAL.COM
>> subject: CN=OCSP Subsystem,O=LOCAL.COM
>> expires: 2017-01-09 08:21:45 UTC
>> key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
>> eku: id-kp-OCSPSigning
>> pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>> post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
>> "ocspSigningCert cert-pki-ca"
>> track: yes
>> auto-renew: yes
>>
>> These were originally in CA_WORKING state, but I moved the clock back
>> and restarted certmonger to try to renew them.
>>
>>
>> /var/log/pki/pki-tomcat/ca/debug contains
>>
>> [10/Jan/2017:18:35:37][localhost-startStop-1]: makeConnection:
>> errorIfDown true
>> [10/Jan/2017:18:35:37][localhost-startStop-1]:
>> SSLClientCertificateSelectionCB: Setting desired cert nickname to:
>> subsystemCert cert-pki-ca
>> [10/Jan/2017:18:35:37][localhost-startStop-1]: LdapJssSSLSocket: set
>> client auth cert nickname subsystemCert cert-pki-ca
>> [10/Jan/2017:18:35:37][localhost-startStop-1]:
>> SSLClientCertificatSelectionCB: Entering!
>> [10/Jan/2017:18:35:37][localhost-startStop-1]: Candidate cert:
>> caSigningCert cert-pki-ca
>> [10/Jan/2017:18:35:37][localhost-startStop-1]: Candidate cert:
>> Server-Cert cert-pki-ca
>> [10/Jan/2017:18:35:37][localhost-startStop-1]:
>> SSLClientCertificateSelectionCB: returning: null
>> [10/Jan/2017:18:35:37][localhost-startStop-1]: SSL handshake happened
>> Could not connect to LDAP server host ipa001.mgmt.local.com port 636
>> Error netscape.ldap.LDAPException: Authentication failed (48)
>> at
>> com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConnection(LdapBoundConnFactory.java:205)
>> at
>> com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(LdapBoundConnFactory.java:166)
>> at
>> com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(LdapBoundConnFactory.java:130)
>> at com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:654)
>> at
>> com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.java:1169)
>> at
>> com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine.java:1075)
>> at com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:571)
>> at com.netscape.certsrv.apps.CMS.init(CMS.java:187)
>> at com.netscape.certsrv.apps.CMS.start(CMS.java:1616)
>> at
>> com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114)
>> at javax.servlet.GenericServlet.init(GenericServlet.java:158)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at
>> org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
>> at
>> org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
>> at
>> org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
>> at
>> org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)
>> at
>> org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:124)
>> at
>> org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1270)
>> at
>> org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1195)
>> at
>> org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1085)
>> at
>> org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5318)
>> at
>> org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5610)
>> at
>> org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
>> at
>> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899)
>> at
>> org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:133)
>> at
>> org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:156)
>> at
>> org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:145)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at
>> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:873)
>> at
>> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)
>> at
>> org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:679)
>> at
>> org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1966)
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Internal Database Error encountered: Could not connect to LDAP server
>> host ipa001.mgmt.local.com port 636 Error netscape.ldap.LDAPException:
>> Authentication failed (48)
>> at com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:676)
>>
>> The only connection attempt I can find relating to err=48 in the slapd
>> access log is -
>>
>>
>> [10/Jan/2017:18:21:08.884446519 +0000] conn=59668 fd=83 slot=83 SSL
>> connection from 10.220.6.250 to 10.220.6.250
>> [10/Jan/2017:18:21:08.898844561 +0000] conn=59668 TLS1.2 256-bit AES
>> [10/Jan/2017:18:21:08.917314723 +0000] conn=59668 op=0 BIND dn=""
>> method=sasl version=3 mech=EXTERNAL
>> [10/Jan/2017:18:21:08.919725280 +0000] conn=59668 op=0 RESULT err=48
>> tag=97 nentries=0 etime=0
>> [10/Jan/2017:18:21:09.590236408 +0000] conn=59637 op=88 EXT
>> oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop"
>>
>> We recent upgraded ipa from 4.2 to 4.4 and I wonder if that broke something.
>>
>> ipa --version
>> VERSION: 4.4.0, API_VERSION: 2.213
>>
>> The /etc/ca.crt cert was originally created on an ipa 3.3 server that no
>> longer exists, I don't know if that's relevant.
>>
>> Anyway, I'm stumped on how to fix this so could anyone please help.
>>
>> Many thanks
>>
>> Bob
>>
>> --
>> Manage your subscription for the Freeipa-users mailing list:
>> https://www.redhat.com/mailman/listinfo/freeipa-users
>> Go to http://freeipa.org for more info on the project
More information about the Freeipa-users
mailing list