<div dir="ltr">Rob/Florence,<div><br></div><div>do you have any pointers on how to troubleshoot, reinstall/configure, update or fix the PKI server to function properly?</div><div>Also if you know of any documentation or video that could be helpful. </div><div>I researched the typical suspects youtube and <a href="http://freeipa.org">freeipa.org</a> without luck.</div><div><br></div><div>Daniel</div></div><div class="gmail_extra"><br><div class="gmail_quote">2016-12-22 18:08 GMT-06:00 Daniel Schimpfoessl <span dir="ltr"><<a href="mailto:daniel@schimpfoessl.com" target="_blank">daniel@schimpfoessl.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I do not believe I changed the DM password. I know I had to update the admin passwords regularly.<div><div><br></div></div><div>Only during the startup using ipactl start --force I am able to connect to the service using the password for DM and it returns:</div><div><br></div><div><div># extended LDIF</div><div>#</div><div># LDAPv3</div><div># base <> with scope baseObject</div><div># filter: (objectclass=*)</div><div># requesting: ALL</div><div>#</div><div><br></div><div>#</div><div>dn:</div><div>objectClass: top</div><div>namingContexts: cn=changelog</div><div>namingContexts: dc=myorg,dc=com</div><div>namingContexts: o=ipaca</div><div>defaultnamingcontext: dc=myorg,dc=com</div><div>supportedExtension: 2.16.840.1.113730.3.5.7</div><div>supportedExtension: 2.16.840.1.113730.3.5.8</div><div>supportedExtension: 2.16.840.1.113730.3.5.10</div><div>supportedExtension: 2.16.840.1.113730.3.8.10.3</div><div>supportedExtension: 2.16.840.1.113730.3.8.10.4</div><div>supportedExtension: 2.16.840.1.113730.3.8.10.4.1</div><div>supportedExtension: 1.3.6.1.4.1.4203.1.11.1</div><div>supportedExtension: 2.16.840.1.113730.3.8.10.1</div><div>supportedExtension: 2.16.840.1.113730.3.8.10.5</div><div>supportedExtension: 2.16.840.1.113730.3.5.3</div><div>supportedExtension: 2.16.840.1.113730.3.5.12</div><div>supportedExtension: 2.16.840.1.113730.3.5.5</div><div>supportedExtension: 2.16.840.1.113730.3.5.6</div><div>supportedExtension: 2.16.840.1.113730.3.5.9</div><div>supportedExtension: 2.16.840.1.113730.3.5.4</div><div>supportedExtension: 2.16.840.1.113730.3.6.5</div><div>supportedExtension: 2.16.840.1.113730.3.6.6</div><div>supportedExtension: 2.16.840.1.113730.3.6.7</div><div>supportedExtension: 2.16.840.1.113730.3.6.8</div><div>supportedExtension: 1.3.6.1.4.1.1466.20037</div><div>supportedControl: 2.16.840.1.113730.3.4.2</div><div>supportedControl: 2.16.840.1.113730.3.4.3</div><div>supportedControl: 2.16.840.1.113730.3.4.4</div><div>supportedControl: 2.16.840.1.113730.3.4.5</div><div>supportedControl: 1.2.840.113556.1.4.473</div><div>supportedControl: 2.16.840.1.113730.3.4.9</div><div>supportedControl: 2.16.840.1.113730.3.4.16</div><div>supportedControl: 2.16.840.1.113730.3.4.15</div><div>supportedControl: 2.16.840.1.113730.3.4.17</div><div>supportedControl: 2.16.840.1.113730.3.4.19</div><div>supportedControl: 1.3.6.1.1.13.1</div><div>supportedControl: 1.3.6.1.1.13.2</div><div>supportedControl: 1.3.6.1.4.1.42.2.27.8.5.1</div><div>supportedControl: 1.3.6.1.4.1.42.2.27.9.5.2</div><div>supportedControl: 1.2.840.113556.1.4.319</div><div>supportedControl: 1.3.6.1.4.1.42.2.27.9.5.8</div><div>supportedControl: 1.3.6.1.4.1.4203.666.5.16</div><div>supportedControl: 2.16.840.1.113730.3.8.10.6</div><div>supportedControl: 2.16.840.1.113730.3.4.14</div><div>supportedControl: 2.16.840.1.113730.3.4.20</div><div>supportedControl: 1.3.6.1.4.1.1466.29539.12</div><div>supportedControl: 2.16.840.1.113730.3.4.12</div><div>supportedControl: 2.16.840.1.113730.3.4.18</div><div>supportedControl: 2.16.840.1.113730.3.4.13</div><div>supportedControl: 1.3.6.1.4.1.4203.1.9.1.1</div><div>supportedSASLMechanisms: EXTERNAL</div><div>supportedSASLMechanisms: GSS-SPNEGO</div><div>supportedSASLMechanisms: GSSAPI</div><div>supportedSASLMechanisms: DIGEST-MD5</div><div>supportedSASLMechanisms: CRAM-MD5</div><div>supportedSASLMechanisms: ANONYMOUS</div><div>supportedLDAPVersion: 2</div><div>supportedLDAPVersion: 3</div><div>vendorName: 389 Project</div><div>vendorVersion: 389-Directory/<a href="http://1.3.4.0" target="_blank">1.3.4.0</a> B2016.215.1556</div><div>dataversion: 020161222235947020161222235947<wbr>020161222235947</div><div>netscapemdsuffix: cn=ldap://dc=wwgwho01,dc=<wbr>myorg,dc=com:389</div><div>lastusn: 8690425</div><div>changeLog: cn=changelog</div><div>firstchangenumber: 2752153</div><div>lastchangenumber: 2752346</div><div><br></div><div># search result</div><div>search: 2</div><div>result: 0 Success</div><div><br></div><div># numResponses: 2</div><div># numEntries: 1</div></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">2016-12-21 9:27 GMT-06:00 Rob Crittenden <span dir="ltr"><<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Daniel Schimpfoessl wrote:<br>
> Thanks for getting back to me.<br>
><br>
> getcert list | grep expires shows dates years in the future for all<br>
> certificates<br>
> Inline-Bild 1<br>
><br>
> ipactl start --force<br>
><br>
> Eventually the system started with:<br>
> Forced start, ignoring pki-tomcatd Service, continuing normal<br>
> operations.<br>
><br>
> systemctl status ipa shows: failed<br>
<br>
I don't think this is a certificate problem at all. I think the timing<br>
with your renewal is just coincidence.<br>
<br>
Did you change your Directory Manager password at some point?<br>
<br>
><br>
> ldapsearch -H ldaps://localhost:636 -D "cn=directory manager" -w<br>
> password -b "" -s base<br>
> ldapsearch -H ldaps://localhost:636 -D "cn=directory manager" -w<br>
> *********** -b "" -s base<br>
> Inline-Bild 2<br>
<br>
You need the -x flag to indicate simple bind.<br>
<br>
rob<br>
<br>
> The logs have thousands of lines like it, what am I looking for<br>
> specifically?<br>
><br>
> Daniel<br>
><br>
><br>
> 2016-12-20 4:18 GMT-06:00 Florence Blanc-Renaud <<a href="mailto:flo@redhat.com" target="_blank">flo@redhat.com</a><br>
> <mailto:<a href="mailto:flo@redhat.com" target="_blank">flo@redhat.com</a>>>:<br>
<span>><br>
> On 12/19/2016 07:15 PM, Daniel Schimpfoessl wrote:<br>
><br>
> Good day and happy holidays,<br>
><br>
> I have been running a freeIPA instance for a few years and been very<br>
> happy. Recently the certificate expired and I updated it using the<br>
> documented methods. At first all seemed fine. Added a Nagios<br>
> monitor for<br>
> the certificate expiration and restarted the server (single<br>
> server). I<br>
> have weekly snapshots, daily backups (using Amanda on the entire<br>
> disk).<br>
><br>
> One day the services relying on IPA failed to authenticate.<br>
> Looking at<br>
> the server the ipa service had stopped. Restarting the service<br>
> fails.<br>
> Restoring a few weeks old snapshot does not start either.<br>
> Resetting the<br>
> date to a few month back does not work either as httpd fails to<br>
> start .<br>
><br>
> I am at a loss.<br>
><br>
</span><span>> Here a few details:<br>
> # ipa --version<br>
> VERSION: 4.4.0, API_VERSION: 2.213<br>
><br>
><br>
> # /usr/sbin/ipactl start<br>
> ...<br>
> out -> Failed to start pki-tomcatd Service<br>
> /var/log/pki/pki-tomcat/ca/de<wbr>bug -> Could not connect to LDAP server<br>
</span>> host <a href="http://ipa.myorg.com" rel="noreferrer" target="_blank">ipa.myorg.com</a> <<a href="http://ipa.myorg.com" rel="noreferrer" target="_blank">http://ipa.myorg.com</a>> <<a href="http://ipa.myorg.com" rel="noreferrer" target="_blank">http://ipa.myorg.com</a>><br>
<span>> port 636 Error<br>
> netscape.ldap.LDAPException: Authentication failed (48)<br>
> 2016-12-19T03:02:16Z DEBUG The CA status is: check interrupted<br>
> due to<br>
> error: Retrieving CA status failed with status 500<br>
><br>
> Any help would be appreciated as all connected services are now<br>
> down.<br>
><br>
> Thanks,<br>
><br>
> Daniel<br>
><br>
><br>
><br>
><br>
</span>> Hi Daniel,<br>
><br>
> more information would be required to understand what is going on.<br>
> First of all, which certificate did you renew? Can you check with<br>
> $ getcert list<br>
> if other certificates also expired?<br>
><br>
> PKI fails to start and the error seems linked to the SSL connection<br>
> with the LDAP server. You may want to check if the LDAP server is<br>
> listening on the LDAPs port:<br>
> - start the stack with<br>
> $ ipactl start --force<br>
> - check the LDAPs port with<br>
> $ ldapsearch -H ldaps://localhost:636 -D "cn=directory manager" -w<br>
> password -b "" -s base<br>
><br>
> The communication between PKI and the LDAP server is authenticated<br>
> with the certificate 'subsystemCert cert-pki-ca' located in<br>
> /etc/pki/pki-tomcat/alias, so you may also want to check if it is<br>
> still valid.<br>
> The directory server access logs (in<br>
> /var/log/dirsrv/slapd-DOMAIN-<wbr>COM/access) would also show the<br>
> connection with logs similar to:<br>
><br>
> [...] conn=47 fd=84 slot=84 SSL connection from 10.34.58.150 to<br>
> 10.34.58.150<br>
> [...] conn=47 TLS1.2 128-bit AES; client CN=CA<br>
> Subsystem,O=<a href="http://DOMAIN.COM" rel="noreferrer" target="_blank">DOMAIN.COM</a> <<a href="http://DOMAIN.COM" rel="noreferrer" target="_blank">http://DOMAIN.COM</a>>; issuer CN=Certificate<br>
> Authority,O=<a href="http://DOMAIN.COM" rel="noreferrer" target="_blank">DOMAIN.COM</a> <<a href="http://DOMAIN.COM" rel="noreferrer" target="_blank">http://DOMAIN.COM</a>><br>
> [...] conn=47 TLS1.2 client bound as uid=pkidbuser,ou=people,o=ipac<wbr>a<br>
> [...] conn=47 op=0 BIND dn="" method=sasl version=3 mech=EXTERNAL<br>
> [...] conn=47 op=0 RESULT err=0 tag=97 nentries=0 etime=0<br>
> dn="uid=pkidbuser,ou=people,<wbr>o=ipaca"<br>
><br>
><br>
><br>
> HTH,<br>
> Flo<br>
><br>
><br>
><br>
><br>
<br>
</blockquote></div><br></div>
</blockquote></div><br></div>