<div dir="ltr">Thank you Rob! I now have two years till everything expires...<div class="gmail_extra"> <div class="gmail_quote">On Tue, Jun 21, 2016 at 1:33 PM, Rob Crittenden <<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">Marc Wiatrowski wrote: <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"> Thanks for the reply Rob, So should fixing replication be more than running a re-initialize? I've tried this with no luck. Still the same errors in renewing the IPA certs. </blockquote> re-init drops one database and replaces it with another. If you really did that then you have potentially lost a ton of records if indeed replication was stalled. Knowing what commands you ran would help to know for sure.</blockquote><div> </div><div> </div><div>I'm thinking at some point in the past I may have done this backwards. So maybe not my original problem but making things worse. </div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"> status: CA_UNREACHABLE ca-error: Server at <a href="https://spider01a.iglass.net/ipa/xml" rel="noreferrer" target="_blank">https://spider01a.iglass.net/ipa/xml</a> failed request, will retry: 4301 (RPC failed at server. Certificate operation cannot be completed: EXCEPTION (Certificate serial number 0x3ffe000f not found)) Is there a procedure for getting these serial numbers back in to the system? or manually recreating somehow? </blockquote> When IPA gets a certificate request and the host/service it is requesting it for already has a certificate, a revocation is done on the existing certificate (which in this case is failing because the cert is unknown). If you wipe out the usercertificate field from the entry ldap/<a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">spider01a.iglass.net</a> then that should do it.</blockquote><div> </div><div> </div><div>This did the trick! I also had to delete userCertificate for dogtagldap/<a href="http://spider01a.iglass.net">spider01a.iglass.net</a> and HTTP/<a href="http://spider01a.iglass.net">spider01a.iglass.net</a> for the other two certificates not renewing.</div><div> </div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"> I was able to clear 4301 error. One ipaCert needed to be updated. </blockquote> Great! rob <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"> thanks On Thu, Jun 16, 2016 at 10:22 AM, Rob Crittenden <<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>>> wrote: Marc Wiatrowski wrote: <div><div class="h5"> Thanks Rob, Any suggestions on how make the CA aware of the current serial number? Serial numbers are dolled out like uid numbers, by the 389-ds DNA Plugin. So each CA that has ever issued a certificate has its own range, hence the quite different serial number values. Given that some issued certificates are unknown it stands to reason that replication is broken between one or more masters. Fixing that should resolve (most of) the other issues. Also started seeing the following error from two of the servers, spider01b and spider01o, but not spider01a when to navigate in the web gui. Though it doesn't appear to stop me from doing anything. IPA Error 4301 Certificate operation cannot be completed: EXCEPTION (Invalid Crential.) Dogtag does some of its access control by comparing the incoming client certificate with an expected value in its LDAP database, in this case uid=ipara,ou=People,o=ipaca. There you'll find a copy of the client certificate and a description field that contains the expected serial #, subject and issuer. These are out-of-whack if you're getting Invalid Credentials. It could be a number of things so I'd proceed cautiously. Given you have a working master I'd use that as a starting point. Look at the the RA cert is in /etc/httpd/alias: # certutil -L -d /etc/httpd/alias -n ipaCert | grep Serial See if it is the same on all masters, it should be. If it is, look at the uid=ipara entry on all the masters. Again, should be the same. Note that fixing this won't address any replication issues. rob Marc On Tue, Jun 14, 2016 at 2:07 PM, Marc Wiatrowski <<a href="mailto:wia@iglass.net" target="_blank">wia@iglass.net</a> <mailto:<a href="mailto:wia@iglass.net" target="_blank">wia@iglass.net</a>> </div></div> <mailto:<a href="mailto:wia@iglass.net" target="_blank">wia@iglass.net</a> <mailto:<a href="mailto:wia@iglass.net" target="_blank">wia@iglass.net</a>>>> wrote: On Tue, Jun 14, 2016 at 11:22 AM, Rob Crittenden <<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>> <div><div class="h5"> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>>>> wrote: Marc Wiatrowski wrote: Hello, I'm having issues with the 3 ipa certificates of type CA: IPA renewing on 2 of 3 replicas. Particularly on the 2 that are not the CA master. The other 5 certificates from getcert list do renew and all certificates on the CA master do look to renew. Both servers running ipa-server-3.0.0-50.el6.centos.1.x86_64 I've done full updates and rebooted. Can you check on the replication status for each CA? $ ipa-csreplica-manage list -v <a href="http://ipa.example.com" rel="noreferrer" target="_blank">ipa.example.com</a> <<a href="http://ipa.example.com" rel="noreferrer" target="_blank">http://ipa.example.com</a>> <<a href="http://ipa.example.com" rel="noreferrer" target="_blank">http://ipa.example.com</a>> The hostname is important because including that will show the agreements that host has. Do this for each master with a CA. The CA being asked to do the renewal is unaware of the current serial number so it is refusing to proceed. rob [root@spider01o]$ ipa-csreplica-manage list -v <a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">spider01a.iglass.net</a> <<a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">http://spider01a.iglass.net</a>> <<a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">http://spider01a.iglass.net</a>> Directory Manager password: <a href="http://spider01b.iglass.net" rel="noreferrer" target="_blank">spider01b.iglass.net</a> <<a href="http://spider01b.iglass.net" rel="noreferrer" target="_blank">http://spider01b.iglass.net</a>> <<a href="http://spider01b.iglass.net" rel="noreferrer" target="_blank">http://spider01b.iglass.net</a>> last init status: None last init ended: None last update status: 0 Replica acquired successfully: Incremental update succeeded last update ended: 2016-06-14 17:49:16+00:00 <a href="http://spider01o.iglass.net" rel="noreferrer" target="_blank">spider01o.iglass.net</a> <<a href="http://spider01o.iglass.net" rel="noreferrer" target="_blank">http://spider01o.iglass.net</a>> <<a href="http://spider01o.iglass.net" rel="noreferrer" target="_blank">http://spider01o.iglass.net</a>> last init status: None last init ended: None last update status: 0 Replica acquired successfully: Incremental update started last update ended: 2016-06-14 17:55:20+00:00 [root@spider01o]$ ipa-csreplica-manage list -v <a href="http://spider01o.iglass.net" rel="noreferrer" target="_blank">spider01o.iglass.net</a> <<a href="http://spider01o.iglass.net" rel="noreferrer" target="_blank">http://spider01o.iglass.net</a>> <<a href="http://spider01o.iglass.net" rel="noreferrer" target="_blank">http://spider01o.iglass.net</a>> Directory Manager password: <a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">spider01a.iglass.net</a> <<a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">http://spider01a.iglass.net</a>> <<a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">http://spider01a.iglass.net</a>> last init status: None last init ended: None last update status: 0 Replica acquired successfully: Incremental update started last update ended: 2016-06-14 17:57:44+00:00 <a href="http://spider01b.iglass.net" rel="noreferrer" target="_blank">spider01b.iglass.net</a> <<a href="http://spider01b.iglass.net" rel="noreferrer" target="_blank">http://spider01b.iglass.net</a>> <<a href="http://spider01b.iglass.net" rel="noreferrer" target="_blank">http://spider01b.iglass.net</a>> last init status: None last init ended: None last update status: 0 Replica acquired successfully: Incremental update started last update ended: 2016-06-14 17:57:41+00:00 [root@spider01o]$ ipa-csreplica-manage list -v <a href="http://spider01b.iglass.net" rel="noreferrer" target="_blank">spider01b.iglass.net</a> <<a href="http://spider01b.iglass.net" rel="noreferrer" target="_blank">http://spider01b.iglass.net</a>> <<a href="http://spider01b.iglass.net" rel="noreferrer" target="_blank">http://spider01b.iglass.net</a>> Directory Manager password: <a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">spider01a.iglass.net</a> <<a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">http://spider01a.iglass.net</a>> <<a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">http://spider01a.iglass.net</a>> last init status: 0 Total update succeeded last init ended: 2016-06-03 19:43:12+00:00 last update status: 0 Replica acquired successfully: Incremental update succeeded last update ended: 2016-06-14 17:44:17+00:00 <a href="http://spider01o.iglass.net" rel="noreferrer" target="_blank">spider01o.iglass.net</a> <<a href="http://spider01o.iglass.net" rel="noreferrer" target="_blank">http://spider01o.iglass.net</a>> <<a href="http://spider01o.iglass.net" rel="noreferrer" target="_blank">http://spider01o.iglass.net</a>> last init status: 0 Total update succeeded last init ended: 2016-06-03 19:44:38+00:00 last update status: 0 Replica acquired successfully: Incremental update started last update ended: 2016-06-14 17:57:53+00:00 <a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">spider01a.iglass.net</a> <<a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">http://spider01a.iglass.net</a>> <<a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">http://spider01a.iglass.net</a>> last init status: None last init ended: None last update status: 0 Replica acquired successfully: Incremental update succeeded last update ended: 2016-06-14 17:44:13+00:00 <a href="http://spider01o.iglass.net" rel="noreferrer" target="_blank">spider01o.iglass.net</a> <<a href="http://spider01o.iglass.net" rel="noreferrer" target="_blank">http://spider01o.iglass.net</a>> <<a href="http://spider01o.iglass.net" rel="noreferrer" target="_blank">http://spider01o.iglass.net</a>> last init status: None last init ended: None last update status: 0 Replica acquired successfully: Incremental update started last update ended: 2016-06-14 17:57:54+00:00 Not sure what this is telling... This an issue with the last being doubled? Thanks The failed renews look like: [root@spider01a]$ getcert list -i 20141202144354 Number of certificates and requests being tracked: 8. Request ID '20141202144354': status: CA_UNREACHABLE ca-error: Server at <a href="https://spider01a.iglass.net/ipa/xml" rel="noreferrer" target="_blank">https://spider01a.iglass.net/ipa/xml</a> failed request, will retry: 4301 (RPC failed at server. Certificate operation cannot be completed: EXCEPTION (Certificate serial number 0x3ffe0010 not found)). stuck: no key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-PKI-IPA',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-PKI-IPA/pwdfile.txt' certificate: type=NSSDB,location='/etc/dirsrv/slapd-PKI-IPA',nickname='Server-Cert',token='NSS Certificate DB' CA: IPA issuer: CN=Certificate Authority,O=<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">IGLASS.NET</a> </div></div> <<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">http://IGLASS.NET</a>> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://iglass.net/</a>> <<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">http://IGLASS.NET</a> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://iglass.net/</a>>> subject: CN=<a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">spider01a.iglass.net</a> <<a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">http://spider01a.iglass.net</a>> <<a href="http://spider01a.iglass.net/" rel="noreferrer" target="_blank">http://spider01a.iglass.net/</a>> <<a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">http://spider01a.iglass.net</a> <<a href="http://spider01a.iglass.net/" rel="noreferrer" target="_blank">http://spider01a.iglass.net/</a>>>,O=<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">IGLASS.NET</a> <<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">http://IGLASS.NET</a>><div><div class="h5"> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://iglass.net/</a>> <<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">http://IGLASS.NET</a> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://iglass.net/</a>>> expires: 2016-12-02 14:38:45 UTC key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: /usr/lib64/ipa/certmonger/restart_dirsrv PKI-IPA track: yes auto-renew: yes [root@spider01a]$ getcert list -i 20141202144616 Number of certificates and requests being tracked: 8. Request ID '20141202144616': status: CA_UNREACHABLE ca-error: Server at <a href="https://spider01a.iglass.net/ipa/xml" rel="noreferrer" target="_blank">https://spider01a.iglass.net/ipa/xml</a> failed request, will retry: 4301 (RPC failed at server. Certificate operation cannot be completed: EXCEPTION (Certificate serial number 0x3ffe000f not found)). stuck: no key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-IGLASS-NET',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-IGLASS-NET/pwdfile.txt' certificate: type=NSSDB,location='/etc/dirsrv/slapd-IGLASS-NET',nickname='Server-Cert',token='NSS Certificate DB' CA: IPA issuer: CN=Certificate Authority,O=<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">IGLASS.NET</a> </div></div> <<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">http://IGLASS.NET</a>> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://iglass.net/</a>> <<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">http://IGLASS.NET</a> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://iglass.net/</a>>> subject: CN=<a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">spider01a.iglass.net</a> <<a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">http://spider01a.iglass.net</a>> <<a href="http://spider01a.iglass.net/" rel="noreferrer" target="_blank">http://spider01a.iglass.net/</a>> <<a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">http://spider01a.iglass.net</a> <<a href="http://spider01a.iglass.net/" rel="noreferrer" target="_blank">http://spider01a.iglass.net/</a>>>,O=<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">IGLASS.NET</a> <<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">http://IGLASS.NET</a>><div><div class="h5"> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://iglass.net/</a>> <<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">http://IGLASS.NET</a> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://iglass.net/</a>>> expires: 2016-12-02 14:38:43 UTC key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: /usr/lib64/ipa/certmonger/restart_dirsrv IGLASS-NET track: yes auto-renew: yes [root@spider01a]$ getcert list -i 20141202144733 Number of certificates and requests being tracked: 8. Request ID '20141202144733': status: CA_UNREACHABLE ca-error: Server at <a href="https://spider01a.iglass.net/ipa/xml" rel="noreferrer" target="_blank">https://spider01a.iglass.net/ipa/xml</a> failed request, will retry: 4301 (RPC failed at server. Certificate operation cannot be completed: EXCEPTION (Certificate serial number 0x3ffe0011 not found)). stuck: no key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt' certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB' CA: IPA issuer: CN=Certificate Authority,O=<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">IGLASS.NET</a> </div></div> <<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">http://IGLASS.NET</a>> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://iglass.net/</a>> <<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">http://IGLASS.NET</a> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://iglass.net/</a>>> subject: CN=<a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">spider01a.iglass.net</a> <<a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">http://spider01a.iglass.net</a>> <<a href="http://spider01a.iglass.net/" rel="noreferrer" target="_blank">http://spider01a.iglass.net/</a>> <<a href="http://spider01a.iglass.net" rel="noreferrer" target="_blank">http://spider01a.iglass.net</a> <<a href="http://spider01a.iglass.net/" rel="noreferrer" target="_blank">http://spider01a.iglass.net/</a>>>,O=<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">IGLASS.NET</a> <<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">http://IGLASS.NET</a>><div><div class="h5"> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://iglass.net/</a>> <<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">http://IGLASS.NET</a> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://iglass.net/</a>>> expires: 2016-12-02 14:38:46 UTC key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save command: /usr/lib64/ipa/certmonger/restart_httpd track: yes auto-renew: yes From [root@spider01a]$ getcert resubmit -i 20141202144354 On the replica issuing the resubmit ==> /var/log/httpd/access_log <== 192.168.176.2 - - [13/Jun/2016:15:49:32 -0400] "POST /ipa/xml HTTP/1.1" 401 1370 ==> /var/log/httpd/error_log <== [Mon Jun 13 15:49:33 2016] [error] ipa: ERROR: ipaserver.plugins.dogtag.ra.get_certificate(): EXCEPTION (Certificate serial number 0x3ffe0010 not found) [Mon Jun 13 15:49:33 2016] [error] ipa: INFO: host/<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>>>>: cert_request(u'MIIDsTCCApkCAQAwNDETMBEGA1UEChMKSUdMQVNTLk5FVDEdMBsGA1UEAxMUc3BpZGVyMDFhLml...UVrN8lbKn17V5COjnj6k0mdbz3KptL0UI/l0BPlFBWGN5MFYaDx2F+y6LWv/aXeu2V4E6LA==', principal=u'dogtagldap/<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>>>>', add=True): CertificateOperationError ==> /var/log/httpd/access_log <== 192.168.176.2 - - [13/Jun/2016:15:49:33 -0400] "POST /ca/agent/ca/displayBySerial HTTP/1.1" 200 262 192.168.176.2 - host/<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>>>> [13/Jun/2016:15:49:32 -0400] "POST /ipa/xml HTTP/1.1" 200 376 ==> /var/log/pki-ca/system <== 2508.TP-Processor6 - [13/Jun/2016:15:49:33 EDT] [3] [3] Servlet caDisplayBySerial: Error encountered in DisplayBySerial. Error Record not found. On the CA master spider01o: ==> /var/log/httpd/access_log <== 192.168.176.2 - - [13/Jun/2016:15:49:33 -0400] "POST /ipa/xml HTTP/1.1" 401 1370 ==> krb5kdc.log <== Jun 13 15:49:34 <a href="http://spider01o.iglass.net" rel="noreferrer" target="_blank">spider01o.iglass.net</a> <<a href="http://spider01o.iglass.net" rel="noreferrer" target="_blank">http://spider01o.iglass.net</a>> <<a href="http://spider01o.iglass.net/" rel="noreferrer" target="_blank">http://spider01o.iglass.net/</a>> <<a href="http://spider01o.iglass.net" rel="noreferrer" target="_blank">http://spider01o.iglass.net</a> <<a href="http://spider01o.iglass.net/" rel="noreferrer" target="_blank">http://spider01o.iglass.net/</a>>> krb5kdc[1963](info): TGS_REQ (4 etypes {18 17 16 23}) 192.168.177.2 <<a href="http://192.168.177.2" rel="noreferrer" target="_blank">http://192.168.177.2</a> <<a href="http://192.168.177.2/" rel="noreferrer" target="_blank">http://192.168.177.2/</a>>>: ISSUE: authtime 1465847372, etypes {rep=18 tkt=18 ses=18}, host/<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>>>> for ldap/<a href="mailto:spider01o.iglass.net@IGLASS.NET" target="_blank">spider01o.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01o.iglass.net@IGLASS.NET" target="_blank">spider01o.iglass.net@IGLASS.NET</a>> <mailto:<a href="mailto:spider01o.iglass.net@IGLASS.NET" target="_blank">spider01o.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01o.iglass.net@IGLASS.NET" target="_blank">spider01o.iglass.net@IGLASS.NET</a>>> <mailto:<a href="mailto:spider01o.iglass.net@IGLASS.NET" target="_blank">spider01o.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01o.iglass.net@IGLASS.NET" target="_blank">spider01o.iglass.net@IGLASS.NET</a>> <mailto:<a href="mailto:spider01o.iglass.net@IGLASS.NET" target="_blank">spider01o.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01o.iglass.net@IGLASS.NET" target="_blank">spider01o.iglass.net@IGLASS.NET</a>>>> ==> /var/log/httpd/error_log <== [Mon Jun 13 15:49:34 2016] [error] ipa: ERROR: ipaserver.plugins.dogtag.ra.get_certificate(): EXCEPTION (Invalid Credential.) [Mon Jun 13 15:49:34 2016] [error] ipa: INFO: host/<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>>>>: cert_request(u'MIIDsTCCApkCAQAwNDETMBEGA1UEChMKSUdMQVNTLk5FVDEdMBsGA1UEAxMUc3BpZGVyMDFhLml...UVrN8lbKn17V5COjnj6k0mdbz3KptL0UI/l0BPlFBWGN5MFYaDx2F+y6LWv/aXeu2V4E6LA==', principal=u'dogtagldap/<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> </div></div> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>>>>', add=True): CertificateOperationError ==> /var/log/httpd/access_log <== 192.168.177.2 - - [13/Jun/2016:15:49:34 -0400] "POST /ca/agent/ca/displayBySerial HTTP/1.1" 200 235 192.168.176.2 - host/<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a> <mailto:<a href="mailto:spider01a.iglass.net@IGLASS.NET" target="_blank">spider01a.iglass.net@IGLASS.NET</a>>>> [13/Jun/2016:15:49:33 -0400] "POST /ipa/xml HTTP/1.1" 200 349 ==> /var/log/pki-ca/system <== 2231.TP-Processor3 - [13/Jun/2016:15:49:34 EDT] [6] [3] Cannot authenticate agent with certificate Serial 0x5ffc0008 Subject DN CN=IPA RA,O=<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">IGLASS.NET</a> <<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">http://IGLASS.NET</a>> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://iglass.net/</a>> <<a href="http://IGLASS.NET" rel="noreferrer" target="_blank">http://IGLASS.NET</a> <<a href="http://iglass.net/" rel="noreferrer" target="_blank">http://iglass.net/</a>>>. Error: User not found I realize they expire at the end of the year, but I've had my certificates expire before and would rather not go through that again. Any idea on what's wrong or suggestions on where to look would be appreciated. Thanks, Marc </blockquote> </blockquote></div> </div></div>