<div dir="ltr"><div>Hello, I'm having issues with the 3 ipa certificates of type CA: IPA renewing on 2 of 3 replicas.  Particularly on the 2 that are not the CA master.  The other 5 certificates from getcert list do renew and all certificates on the CA master do look to renew.</div><div><br></div><div>Both servers running ipa-server-3.0.0-50.el6.centos.1.x86_64  I've done full updates and rebooted.<br></div><div><br></div><div>The failed renews look like:</div><div><br></div><div><div>[root@spider01a]$ getcert list -i 20141202144354</div><div>Number of certificates and requests being tracked: 8.</div><div>Request ID '20141202144354':</div><div><span class="" style="white-space:pre">        </span>status: CA_UNREACHABLE</div><div><span class="" style="white-space:pre">     </span>ca-error: Server at <a href="https://spider01a.iglass.net/ipa/xml">https://spider01a.iglass.net/ipa/xml</a> failed request, will retry: 4301 (RPC failed at server.  Certificate operation cannot be completed: EXCEPTION (Certificate serial number 0x3ffe0010 not found)).</div><div><span class="" style="white-space:pre">        </span>stuck: no</div><div><span class="" style="white-space:pre">  </span>key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-PKI-IPA',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-PKI-IPA/pwdfile.txt'</div><div><span class="" style="white-space:pre">        </span>certificate: type=NSSDB,location='/etc/dirsrv/slapd-PKI-IPA',nickname='Server-Cert',token='NSS Certificate DB'</div><div><span class="" style="white-space:pre">     </span>CA: IPA</div><div><span class="" style="white-space:pre">    </span>issuer: CN=Certificate Authority,O=<a href="http://IGLASS.NET">IGLASS.NET</a></div><div><span class="" style="white-space:pre">        </span>subject: CN=<a href="http://spider01a.iglass.net">spider01a.iglass.net</a>,O=<a href="http://IGLASS.NET">IGLASS.NET</a></div><div><span class="" style="white-space:pre">        </span>expires: 2016-12-02 14:38:45 UTC</div><div><span class="" style="white-space:pre">   </span>key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment</div><div><span class="" style="white-space:pre">        </span>eku: id-kp-serverAuth,id-kp-clientAuth</div><div><span class="" style="white-space:pre">     </span>pre-save command: </div><div><span class="" style="white-space:pre">        </span>post-save command: /usr/lib64/ipa/certmonger/restart_dirsrv PKI-IPA</div><div><span class="" style="white-space:pre">        </span>track: yes</div><div><span class="" style="white-space:pre"> </span>auto-renew: yes</div><div><br></div><div>[root@spider01a]$ getcert list -i 20141202144616</div><div>Number of certificates and requests being tracked: 8.</div><div>Request ID '20141202144616':</div><div><span class="" style="white-space:pre">     </span>status: CA_UNREACHABLE</div><div><span class="" style="white-space:pre">     </span>ca-error: Server at <a href="https://spider01a.iglass.net/ipa/xml">https://spider01a.iglass.net/ipa/xml</a> failed request, will retry: 4301 (RPC failed at server.  Certificate operation cannot be completed: EXCEPTION (Certificate serial number 0x3ffe000f not found)).</div><div><span class="" style="white-space:pre">        </span>stuck: no</div><div><span class="" style="white-space:pre">  </span>key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-IGLASS-NET',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-IGLASS-NET/pwdfile.txt'</div><div><span class="" style="white-space:pre">  </span>certificate: type=NSSDB,location='/etc/dirsrv/slapd-IGLASS-NET',nickname='Server-Cert',token='NSS Certificate DB'</div><div><span class="" style="white-space:pre">  </span>CA: IPA</div><div><span class="" style="white-space:pre">    </span>issuer: CN=Certificate Authority,O=<a href="http://IGLASS.NET">IGLASS.NET</a></div><div><span class="" style="white-space:pre">        </span>subject: CN=<a href="http://spider01a.iglass.net">spider01a.iglass.net</a>,O=<a href="http://IGLASS.NET">IGLASS.NET</a></div><div><span class="" style="white-space:pre">        </span>expires: 2016-12-02 14:38:43 UTC</div><div><span class="" style="white-space:pre">   </span>key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment</div><div><span class="" style="white-space:pre">        </span>eku: id-kp-serverAuth,id-kp-clientAuth</div><div><span class="" style="white-space:pre">     </span>pre-save command: </div><div><span class="" style="white-space:pre">        </span>post-save command: /usr/lib64/ipa/certmonger/restart_dirsrv IGLASS-NET</div><div><span class="" style="white-space:pre">     </span>track: yes</div><div><span class="" style="white-space:pre"> </span>auto-renew: yes</div><div><br></div><div>[root@spider01a]$ getcert list -i 20141202144733</div><div>Number of certificates and requests being tracked: 8.</div><div>Request ID '20141202144733':</div><div><span class="" style="white-space:pre">     </span>status: CA_UNREACHABLE</div><div><span class="" style="white-space:pre">     </span>ca-error: Server at <a href="https://spider01a.iglass.net/ipa/xml">https://spider01a.iglass.net/ipa/xml</a> failed request, will retry: 4301 (RPC failed at server.  Certificate operation cannot be completed: EXCEPTION (Certificate serial number 0x3ffe0011 not found)).</div><div><span class="" style="white-space:pre">        </span>stuck: no</div><div><span class="" style="white-space:pre">  </span>key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'</div><div><span class="" style="white-space:pre">  </span>certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB'</div><div><span class="" style="white-space:pre">      </span>CA: IPA</div><div><span class="" style="white-space:pre">    </span>issuer: CN=Certificate Authority,O=<a href="http://IGLASS.NET">IGLASS.NET</a></div><div><span class="" style="white-space:pre">        </span>subject: CN=<a href="http://spider01a.iglass.net">spider01a.iglass.net</a>,O=<a href="http://IGLASS.NET">IGLASS.NET</a></div><div><span class="" style="white-space:pre">        </span>expires: 2016-12-02 14:38:46 UTC</div><div><span class="" style="white-space:pre">   </span>key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment</div><div><span class="" style="white-space:pre">        </span>eku: id-kp-serverAuth,id-kp-clientAuth</div><div><span class="" style="white-space:pre">     </span>pre-save command: </div><div><span class="" style="white-space:pre">        </span>post-save command: /usr/lib64/ipa/certmonger/restart_httpd</div><div><span class="" style="white-space:pre"> </span>track: yes</div><div><span class="" style="white-space:pre"> </span>auto-renew: yes</div></div><div><br></div><div><br></div><div>From</div><div>[root@spider01a]$ getcert resubmit -i 20141202144354<br></div><div><br></div><div>On the replica issuing the resubmit</div><div><br></div><div><div>==> /var/log/httpd/access_log <==</div><div>192.168.176.2 - - [13/Jun/2016:15:49:32 -0400] "POST /ipa/xml HTTP/1.1" 401 1370</div><div><br></div><div>==> /var/log/httpd/error_log <==</div><div>[Mon Jun 13 15:49:33 2016] [error] ipa: ERROR: ipaserver.plugins.dogtag.ra.get_certificate(): EXCEPTION (Certificate serial number 0x3ffe0010 not found)</div><div>[Mon Jun 13 15:49:33 2016] [error] ipa: INFO: host/<a href="mailto:spider01a.iglass.net@IGLASS.NET">spider01a.iglass.net@IGLASS.NET</a>: cert_request(u'MIIDsTCCApkCAQAwNDETMBEGA1UEChMKSUdMQVNTLk5FVDEdMBsGA1UEAxMUc3BpZGVyMDFhLml...UVrN8lbKn17V5COjnj6k0mdbz3KptL0UI/l0BPlFBWGN5MFYaDx2F+y6LWv/aXeu2V4E6LA==', principal=u'dogtagldap/<a href="mailto:spider01a.iglass.net@IGLASS.NET">spider01a.iglass.net@IGLASS.NET</a>', add=True): CertificateOperationError</div><div><br></div><div>==> /var/log/httpd/access_log <==</div><div>192.168.176.2 - - [13/Jun/2016:15:49:33 -0400] "POST /ca/agent/ca/displayBySerial HTTP/1.1" 200 262</div><div>192.168.176.2 - host/<a href="mailto:spider01a.iglass.net@IGLASS.NET">spider01a.iglass.net@IGLASS.NET</a> [13/Jun/2016:15:49:32 -0400] "POST /ipa/xml HTTP/1.1" 200 376</div><div><br></div><div>==> /var/log/pki-ca/system <==</div><div>2508.TP-Processor6 - [13/Jun/2016:15:49:33 EDT] [3] [3] Servlet caDisplayBySerial: Error encountered in DisplayBySerial. Error Record not found.</div></div><div><br></div><div><br></div><div>On the CA master spider01o:</div><div><br></div><div><div>==> /var/log/httpd/access_log <==</div><div>192.168.176.2 - - [13/Jun/2016:15:49:33 -0400] "POST /ipa/xml HTTP/1.1" 401 1370</div><div><br></div><div>==> krb5kdc.log <==</div><div>Jun 13 15:49:34 <a href="http://spider01o.iglass.net">spider01o.iglass.net</a> krb5kdc[1963](info): TGS_REQ (4 etypes {18 17 16 23}) <a href="http://192.168.177.2">192.168.177.2</a>: ISSUE: authtime 1465847372, etypes {rep=18 tkt=18 ses=18}, host/<a href="mailto:spider01a.iglass.net@IGLASS.NET">spider01a.iglass.net@IGLASS.NET</a> for ldap/<a href="mailto:spider01o.iglass.net@IGLASS.NET">spider01o.iglass.net@IGLASS.NET</a></div><div><br></div><div>==> /var/log/httpd/error_log <==</div><div>[Mon Jun 13 15:49:34 2016] [error] ipa: ERROR: ipaserver.plugins.dogtag.ra.get_certificate(): EXCEPTION (Invalid Credential.)</div><div>[Mon Jun 13 15:49:34 2016] [error] ipa: INFO: host/<a href="mailto:spider01a.iglass.net@IGLASS.NET">spider01a.iglass.net@IGLASS.NET</a>: cert_request(u'MIIDsTCCApkCAQAwNDETMBEGA1UEChMKSUdMQVNTLk5FVDEdMBsGA1UEAxMUc3BpZGVyMDFhLml...UVrN8lbKn17V5COjnj6k0mdbz3KptL0UI/l0BPlFBWGN5MFYaDx2F+y6LWv/aXeu2V4E6LA==', principal=u'dogtagldap/<a href="mailto:spider01a.iglass.net@IGLASS.NET">spider01a.iglass.net@IGLASS.NET</a>', add=True): CertificateOperationError</div><div><br></div><div>==> /var/log/httpd/access_log <==</div><div>192.168.177.2 - - [13/Jun/2016:15:49:34 -0400] "POST /ca/agent/ca/displayBySerial HTTP/1.1" 200 235</div><div>192.168.176.2 - host/<a href="mailto:spider01a.iglass.net@IGLASS.NET">spider01a.iglass.net@IGLASS.NET</a> [13/Jun/2016:15:49:33 -0400] "POST /ipa/xml HTTP/1.1" 200 349</div><div><br></div><div>==> /var/log/pki-ca/system <==</div><div>2231.TP-Processor3 - [13/Jun/2016:15:49:34 EDT] [6] [3] Cannot authenticate agent with certificate Serial 0x5ffc0008 Subject DN CN=IPA RA,O=<a href="http://IGLASS.NET">IGLASS.NET</a>. Error: User not found</div></div><div><br></div><div><br></div><div>I realize they expire at the end of the year, but I've had my certificates expire before and would rather not go through that again.  Any idea on what's wrong or suggestions on where to look would be appreciated.</div><div><br></div><div>Thanks,</div><div>Marc</div><div><br></div></div>