<div dir="ltr"><div>Thanks Rob,</div><div><br></div><div>Any suggestions on how make the CA aware of the current serial number?</div><div><br></div>Also started seeing the following error from two of the servers, spider01b and spider01o, but not spider01a when to navigate in the web gui. Though it doesn't appear to stop me from doing anything.<div><br></div><div>IPA Error 4301</div><div>Certificate operation cannot be completed: EXCEPTION (Invalid Crential.)</div><div><br></div><div>Marc<br><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jun 14, 2016 at 2:07 PM, Marc Wiatrowski <span dir="ltr"><<a href="mailto:wia@iglass.net" target="_blank">wia@iglass.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span class="">On Tue, Jun 14, 2016 at 11:22 AM, Rob Crittenden <span dir="ltr"><<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><span>Marc Wiatrowski wrote:<br> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"> Hello, I'm having issues with the 3 ipa certificates of type CA: IPA<br> renewing on 2 of 3 replicas. Particularly on the 2 that are not the CA<br> master. The other 5 certificates from getcert list do renew and all<br> certificates on the CA master do look to renew.<br> <br> Both servers running ipa-server-3.0.0-50.el6.centos.1.x86_64 I've done<br> full updates and rebooted.<br> </blockquote> <br></span> Can you check on the replication status for each CA?<br> <br> $ ipa-csreplica-manage list -v <a href="http://ipa.example.com" rel="noreferrer" target="_blank">ipa.example.com</a><br> <br> The hostname is important because including that will show the agreements that host has. Do this for each master with a CA.<br> <br> The CA being asked to do the renewal is unaware of the current serial number so it is refusing to proceed.<br> <br> rob<br> <br></blockquote><div><br></div><div><br></div></span><div><div>[root@spider01o]$ ipa-csreplica-manage list -v <a href="http://spider01a.iglass.net" target="_blank">spider01a.iglass.net</a></div><div>Directory Manager password: </div><div><br></div><div><a href="http://spider01b.iglass.net" target="_blank">spider01b.iglass.net</a></div><div> last init status: None</div><div> last init ended: None</div><div> last update status: 0 Replica acquired successfully: Incremental update succeeded</div><div> last update ended: 2016-06-14 17:49:16+00:00</div><div><a href="http://spider01o.iglass.net" target="_blank">spider01o.iglass.net</a></div><div> last init status: None</div><div> last init ended: None</div><div> last update status: 0 Replica acquired successfully: Incremental update started</div><div> last update ended: 2016-06-14 17:55:20+00:00</div><div><br></div><div>[root@spider01o]$ ipa-csreplica-manage list -v <a href="http://spider01o.iglass.net" target="_blank">spider01o.iglass.net</a></div><div>Directory Manager password: </div><div><br></div><div><a href="http://spider01a.iglass.net" target="_blank">spider01a.iglass.net</a></div><div> last init status: None</div><div> last init ended: None</div><div> last update status: 0 Replica acquired successfully: Incremental update started</div><div> last update ended: 2016-06-14 17:57:44+00:00</div><div><a href="http://spider01b.iglass.net" target="_blank">spider01b.iglass.net</a></div><div> last init status: None</div><div> last init ended: None</div><div> last update status: 0 Replica acquired successfully: Incremental update started</div><div> last update ended: 2016-06-14 17:57:41+00:00</div><div><br></div><div>[root@spider01o]$ ipa-csreplica-manage list -v <a href="http://spider01b.iglass.net" target="_blank">spider01b.iglass.net</a></div><div>Directory Manager password: </div><div><br></div><div><a href="http://spider01a.iglass.net" target="_blank">spider01a.iglass.net</a></div><div> last init status: 0 Total update succeeded</div><div> last init ended: 2016-06-03 19:43:12+00:00</div><div> last update status: 0 Replica acquired successfully: Incremental update succeeded</div><div> last update ended: 2016-06-14 17:44:17+00:00</div><div><a href="http://spider01o.iglass.net" target="_blank">spider01o.iglass.net</a></div><div> last init status: 0 Total update succeeded</div><div> last init ended: 2016-06-03 19:44:38+00:00</div><div> last update status: 0 Replica acquired successfully: Incremental update started</div><div> last update ended: 2016-06-14 17:57:53+00:00</div><div><a href="http://spider01a.iglass.net" target="_blank">spider01a.iglass.net</a></div><div> last init status: None</div><div> last init ended: None</div><div> last update status: 0 Replica acquired successfully: Incremental update succeeded</div><div> last update ended: 2016-06-14 17:44:13+00:00</div><div><a href="http://spider01o.iglass.net" target="_blank">spider01o.iglass.net</a></div><div> last init status: None</div><div> last init ended: None</div><div> last update status: 0 Replica acquired successfully: Incremental update started</div><div> last update ended: 2016-06-14 17:57:54+00:00</div></div><div><br></div><div><br></div><div>Not sure what this is telling... This an issue with the last being doubled? Thanks</div><div><br></div></div></div></div> </blockquote></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><br class=""><br>The failed renews look like:<br><br>[root@spider01a]$ getcert list -i 20141202144354<br>Number of certificates and requests being tracked: 8.<br>Request ID '20141202144354':<br>status: CA_UNREACHABLE<br>ca-error: Server at <a href="https://spider01a.iglass.net/ipa/xml" rel="noreferrer" target="_blank">https://spider01a.iglass.net/ipa/xml</a> failed request,<br>will retry: 4301 (RPC failed at server. Certificate operation cannot be<br>completed: EXCEPTION (Certificate serial number 0x3ffe0010 not found)).<br>stuck: no<br>key pair storage:<br>type=NSSDB,location='/etc/dirsrv/slapd-PKI-IPA',nickname='Server-Cert',token='NSS<br>Certificate DB',pinfile='/etc/dirsrv/slapd-PKI-IPA/pwdfile.txt'<br>certificate:<br>type=NSSDB,location='/etc/dirsrv/slapd-PKI-IPA',nickname='Server-Cert',token='NSS<br>Certificate DB'<br>CA: IPA<br>issuer: CN=Certificate Authority,O=<a href="http://iglass.net/" rel="noreferrer" target="_blank">IGLASS.NET</a> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://IGLASS.NET</a>><br>subject: CN=<a href="http://spider01a.iglass.net/" rel="noreferrer" target="_blank">spider01a.iglass.net</a><br><<a href="http://spider01a.iglass.net/" rel="noreferrer" target="_blank">http://spider01a.iglass.net</a>>,O=<a href="http://iglass.net/" rel="noreferrer" target="_blank">IGLASS.NET</a> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://IGLASS.NET</a>><br>expires: 2016-12-02 14:38:45 UTC<br>key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment<br>eku: id-kp-serverAuth,id-kp-clientAuth<br>pre-save command:<br>post-save command: /usr/lib64/ipa/certmonger/restart_dirsrv PKI-IPA<br>track: yes<br>auto-renew: yes<br><br>[root@spider01a]$ getcert list -i 20141202144616<br>Number of certificates and requests being tracked: 8.<br>Request ID '20141202144616':<br>status: CA_UNREACHABLE<br>ca-error: Server at <a href="https://spider01a.iglass.net/ipa/xml" rel="noreferrer" target="_blank">https://spider01a.iglass.net/ipa/xml</a> failed request,<br>will retry: 4301 (RPC failed at server. Certificate operation cannot be<br>completed: EXCEPTION (Certificate serial number 0x3ffe000f not found)).<br>stuck: no<br>key pair storage:<br>type=NSSDB,location='/etc/dirsrv/slapd-IGLASS-NET',nickname='Server-Cert',token='NSS<br>Certificate DB',pinfile='/etc/dirsrv/slapd-IGLASS-NET/pwdfile.txt'<br>certificate:<br>type=NSSDB,location='/etc/dirsrv/slapd-IGLASS-NET',nickname='Server-Cert',token='NSS<br>Certificate DB'<br>CA: IPA<br>issuer: CN=Certificate Authority,O=<a href="http://iglass.net/" rel="noreferrer" target="_blank">IGLASS.NET</a> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://IGLASS.NET</a>><br>subject: CN=<a href="http://spider01a.iglass.net/" rel="noreferrer" target="_blank">spider01a.iglass.net</a><br><<a href="http://spider01a.iglass.net/" rel="noreferrer" target="_blank">http://spider01a.iglass.net</a>>,O=<a href="http://iglass.net/" rel="noreferrer" target="_blank">IGLASS.NET</a> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://IGLASS.NET</a>><br>expires: 2016-12-02 14:38:43 UTC<br>key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment<br>eku: id-kp-serverAuth,id-kp-clientAuth<br>pre-save command:<br>post-save command: /usr/lib64/ipa/certmonger/restart_dirsrv IGLASS-NET<br>track: yes<br>auto-renew: yes<br><br>[root@spider01a]$ getcert list -i 20141202144733<br>Number of certificates and requests being tracked: 8.<br>Request ID '20141202144733':<br>status: CA_UNREACHABLE<br>ca-error: Server at <a href="https://spider01a.iglass.net/ipa/xml" rel="noreferrer" target="_blank">https://spider01a.iglass.net/ipa/xml</a> failed request,<br>will retry: 4301 (RPC failed at server. Certificate operation cannot be<br>completed: EXCEPTION (Certificate serial number 0x3ffe0011 not found)).<br>stuck: no<br>key pair storage:<br>type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS<br>Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'<br>certificate:<br>type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS<br>Certificate DB'<br>CA: IPA<br>issuer: CN=Certificate Authority,O=<a href="http://iglass.net/" rel="noreferrer" target="_blank">IGLASS.NET</a> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://IGLASS.NET</a>><br>subject: CN=<a href="http://spider01a.iglass.net/" rel="noreferrer" target="_blank">spider01a.iglass.net</a><br><<a href="http://spider01a.iglass.net/" rel="noreferrer" target="_blank">http://spider01a.iglass.net</a>>,O=<a href="http://iglass.net/" rel="noreferrer" target="_blank">IGLASS.NET</a> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://IGLASS.NET</a>><br>expires: 2016-12-02 14:38:46 UTC<br>key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment<br>eku: id-kp-serverAuth,id-kp-clientAuth<br>pre-save command:<br>post-save command: /usr/lib64/ipa/certmonger/restart_httpd<br>track: yes<br>auto-renew: yes<br><br><br>From<br>[root@spider01a]$ getcert resubmit -i 20141202144354<br><br>On the replica issuing the resubmit<br><br>==> /var/log/httpd/access_log <==<br>192.168.176.2 - - [13/Jun/2016:15:49:32 -0400] "POST /ipa/xml HTTP/1.1"<br>401 1370<br><br>==> /var/log/httpd/error_log <==<br>[Mon Jun 13 15:49:33 2016] [error] ipa: ERROR:<br>ipaserver.plugins.dogtag.ra.get_certificate(): EXCEPTION (Certificate<br>serial number 0x3ffe0010 not found)<br>[Mon Jun 13 15:49:33 2016] [error] ipa: INFO:<br>host/<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a><br><mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>>:<br>cert_request(u'MIIDsTCCApkCAQAwNDETMBEGA1UEChMKSUdMQVNTLk5FVDEdMBsGA1UEAxMUc3BpZGVyMDFhLml...UVrN8lbKn17V5COjnj6k0mdbz3KptL0UI/l0BPlFBWGN5MFYaDx2F+y6LWv/aXeu2V4E6LA==',<br>principal=u'dogtagldap/<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a><br><mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>>', add=True):<br>CertificateOperationError<br><br>==> /var/log/httpd/access_log <==<br>192.168.176.2 - - [13/Jun/2016:15:49:33 -0400] "POST<br>/ca/agent/ca/displayBySerial HTTP/1.1" 200 262<br>192.168.176.2 - host/<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a><br><mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>> [13/Jun/2016:15:49:32 -0400]<br>"POST /ipa/xml HTTP/1.1" 200 376<br><br>==> /var/log/pki-ca/system <==<br>2508.TP-Processor6 - [13/Jun/2016:15:49:33 EDT] [3] [3] Servlet<br>caDisplayBySerial: Error encountered in DisplayBySerial. Error Record<br>not found.<br><br><br>On the CA master spider01o:<br><br>==> /var/log/httpd/access_log <==<br>192.168.176.2 - - [13/Jun/2016:15:49:33 -0400] "POST /ipa/xml HTTP/1.1"<br>401 1370<br><br>==> krb5kdc.log <==<br>Jun 13 15:49:34 <a href="http://spider01o.iglass.net/" rel="noreferrer" target="_blank">spider01o.iglass.net</a> <<a href="http://spider01o.iglass.net/" rel="noreferrer" target="_blank">http://spider01o.iglass.net</a>><br>krb5kdc[1963](info): TGS_REQ (4 etypes {18 17 16 23}) 192.168.177.2<br><<a href="http://192.168.177.2/" rel="noreferrer" target="_blank">http://192.168.177.2</a>>: ISSUE: authtime 1465847372, etypes {rep=18<br>tkt=18 ses=18}, host/<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a><br><mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>> for<br>ldap/<a href="mailto:spider01o.iglass.net@IGLASS.NET" target="_blank">spider01o.iglass.net@IGLASS.NET</a><br><mailto:<a href="mailto:spider01o.iglass.net@IGLASS.NET" target="_blank">spider01o.iglass.net@IGLASS.NET</a>><br><br>==> /var/log/httpd/error_log <==<br>[Mon Jun 13 15:49:34 2016] [error] ipa: ERROR:<br>ipaserver.plugins.dogtag.ra.get_certificate(): EXCEPTION (Invalid<br>Credential.)<br>[Mon Jun 13 15:49:34 2016] [error] ipa: INFO:<br>host/<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a><br><mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>>:<br>cert_request(u'MIIDsTCCApkCAQAwNDETMBEGA1UEChMKSUdMQVNTLk5FVDEdMBsGA1UEAxMUc3BpZGVyMDFhLml...UVrN8lbKn17V5COjnj6k0mdbz3KptL0UI/l0BPlFBWGN5MFYaDx2F+y6LWv/aXeu2V4E6LA==',<br>principal=u'dogtagldap/<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a><br><mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>>', add=True):<br>CertificateOperationError<br><br>==> /var/log/httpd/access_log <==<br>192.168.177.2 - - [13/Jun/2016:15:49:34 -0400] "POST<br>/ca/agent/ca/displayBySerial HTTP/1.1" 200 235<br>192.168.176.2 - host/<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a><br><mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>> [13/Jun/2016:15:49:33 -0400]<br>"POST /ipa/xml HTTP/1.1" 200 349<br><br>==> /var/log/pki-ca/system <==<br>2231.TP-Processor3 - [13/Jun/2016:15:49:34 EDT] [6] [3] Cannot<br>authenticate agent with certificate Serial 0x5ffc0008 Subject DN CN=IPA<br>RA,O=<a href="http://iglass.net/" rel="noreferrer" target="_blank">IGLASS.NET</a> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://IGLASS.NET</a>>. Error: User not found<br><br><br>I realize they expire at the end of the year, but I've had my<br>certificates expire before and would rather not go through that again.<br>Any idea on what's wrong or suggestions on where to look would be<br>appreciated.<br><br>Thanks,<br>Marc<br><br><br><br></blockquote><br></div></div></div>