[Freeipa-users] ipa fails to start hangs on pki-tomcatd

[Rob Crittenden]

Rob Verduijn wrote:
> Hello,
> 
> For some reason my ipa server no longer boots.
> It keeps trying to start pki-tomcat service.
> 
> Does anybody know where I should start looking to get this fixed ?
> 
> Rob Verduijn
> 
> ipactl -d start gives this output:
> ipa: DEBUG: The CA status is: check interrupted due to error: Command
> ''/usr/bin/wget' '-S' '-O' '-' '--timeout=30' '--no-check-certificate'
> 'https://freeipa02.tjako.thuis:8443/ca/admin/ca/getStatus'' returned
> non-zero exit status 8
> ipa: DEBUG: Waiting for CA to start...
> ipa: DEBUG: Starting external process
> ipa: DEBUG: args='/usr/bin/wget' '-S' '-O' '-' '--timeout=30'
> '--no-check-certificate'
> 'https://freeipa02.tjako.thuis:8443/ca/admin/ca/getStatus'
> ipa: DEBUG: Process finished, return code=8
> ipa: DEBUG: stdout=
> ipa: DEBUG: stderr=--2016-12-01 11:06:12-- 
> https://freeipa02.tjako.thuis:8443/ca/admin/ca/getStatus
> Resolving freeipa02.tjako.thuis (freeipa02.tjako.thuis)... 172.16.1.13
> Connecting to freeipa02.tjako.thuis
> (freeipa02.tjako.thuis)|172.16.1.13|:8443... connected.
> HTTP request sent, awaiting response...
>   HTTP/1.1 500 Internal Server Error
>   Server: Apache-Coyote/1.1
>   Content-Type: text/html;charset=utf-8
>   Content-Language: en
>   Content-Length: 2134
>   Date: Thu, 01 Dec 2016 10:06:13 GMT
>   Connection: close
> 2016-12-01 11:06:13 ERROR 500: Internal Server Error.
> 
> There are also some java warnings in the logs, but its java and I can
> never tell if its a serious error when java gives a warning.
> Dec  1 09:53:59 freeipa02 server: Dec 01, 2016 9:53:59 AM
> org.apache.catalina.startup.SetAllPropertiesRule begin
> Dec  1 09:53:59 freeipa02 server: WARNING:
> [SetAllPropertiesRule]{Server/Service/Connector} Setting property
> 'serverCertNickFile' to
> '/var/lib/pki/pki-tomcat/conf/serverCertNick.conf' did not find a
> matching property.
> Dec  1 09:53:59 freeipa02 server: Dec 01, 2016 9:53:59 AM
> org.apache.catalina.startup.SetAllPropertiesRule begin
> Dec  1 09:53:59 freeipa02 server: WARNING:
> [SetAllPropertiesRule]{Server/Service/Connector} Setting property
> 'passwordFile' to '/var/lib/pki/pki-tomcat/conf/password.conf' did not
> find a matching property.
> Dec  1 09:53:59 freeipa02 server: Dec 01, 2016 9:53:59 AM
> org.apache.catalina.startup.SetAllPropertiesRule begin
> Dec  1 09:53:59 freeipa02 server: WARNING:
> [SetAllPropertiesRule]{Server/Service/Connector} Setting property
> 'passwordClass' to 'org.apache.tomcat.util.net.jss.PlainPasswordFile'
> did not find a matching property.
> Dec  1 09:53:59 freeipa02 server: Dec 01, 2016 9:53:59 AM
> org.apache.catalina.startup.SetAllPropertiesRule begin
> Dec  1 09:53:59 freeipa02 server: WARNING:
> [SetAllPropertiesRule]{Server/Service/Connector} Setting property
> 'certdbDir' to '/var/lib/pki/pki-tomcat/alias' did not find a matching
> property.
> Dec  1 09:53:59 freeipa02 server: Dec 01, 2016 9:53:59 AM
> org.apache.tomcat.util.digester.SetPropertiesRule begin
> Dec  1 09:53:59 freeipa02 server: WARNING:
> [SetPropertiesRule]{Server/Service/Engine/Host} Setting property
> 'xmlValidation' to 'false' did not find a matching property.
> Dec  1 09:53:59 freeipa02 server: Dec 01, 2016 9:53:59 AM
> org.apache.tomcat.util.digester.SetPropertiesRule begin
> Dec  1 09:53:59 freeipa02 server: WARNING:
> [SetPropertiesRule]{Server/Service/Engine/Host} Setting property
> 'xmlNamespaceAware' to 'false' did not find a matching property.
> 
> 
> I'm running centos7.2 x86_64 with the latest patches applied.
> some package versions below
> rpm -qa|egrep "ipa|tomcat"|sort
> ipa-admintools-4.2.0-15.0.1.el7.centos.19.x86_64
> ipa-client-4.2.0-15.0.1.el7.centos.19.x86_64
> ipa-python-4.2.0-15.0.1.el7.centos.19.x86_64
> ipa-server-4.2.0-15.0.1.el7.centos.19.x86_64
> ipa-server-dns-4.2.0-15.0.1.el7.centos.19.x86_64
> libipa_hbac-1.13.0-40.el7_2.12.x86_64
> python-iniparse-0.4-9.el7.noarch
> python-libipa_hbac-1.13.0-40.el7_2.12.x86_64
> sssd-ipa-1.13.0-40.el7_2.12.x86_64
> tomcat-7.0.54-8.el7_2.noarch
> tomcat-el-2.2-api-7.0.54-8.el7_2.noarch
> tomcat-jsp-2.2-api-7.0.54-8.el7_2.noarch
> tomcatjss-7.1.2-1.el7.noarch
> tomcat-lib-7.0.54-8.el7_2.noarch
> tomcat-servlet-3.0-api-7.0.54-8.el7_2.noarch

The debug log is quite verbose. I find it helpful to note where the
previous log ended, starting and pulling the difference and going line
by line. It sometimes fails in one place which cascades to others this
generally makes it hard to grok.

I'd also run `getcert list` and check to ensure that the CA subsystem
certificates are still valid.

rob