[Freeipa-users] CA: IPA certificates not renewing
Marc Wiatrowski
wia at iglass.net
Wed Jun 22 14:10:26 UTC 2016
Thank you Rob! I now have two years till everything expires...
On Tue, Jun 21, 2016 at 1:33 PM, Rob Crittenden <rcritten at redhat.com> wrote:
> Marc Wiatrowski wrote:
>
>> Thanks for the reply Rob,
>>
>> So should fixing replication be more than running a re-initialize?
>> I've tried this with no luck. Still the same errors in renewing the IPA
>> certs.
>>
>
> re-init drops one database and replaces it with another. If you really did
> that then you have potentially lost a ton of records if indeed replication
> was stalled. Knowing what commands you ran would help to know for sure.
I'm thinking at some point in the past I may have done this backwards. So
maybe not my original problem but making things worse.
>
>
> status: CA_UNREACHABLE
>> ca-error: Server at https://spider01a.iglass.net/ipa/xml failed request,
>> will retry: 4301 (RPC failed at server. Certificate operation cannot be
>> completed: EXCEPTION (Certificate serial number 0x3ffe000f not found))
>>
>> Is there a procedure for getting these serial numbers back in to the
>> system? or manually recreating somehow?
>>
>
> When IPA gets a certificate request and the host/service it is requesting
> it for already has a certificate, a revocation is done on the existing
> certificate (which in this case is failing because the cert is unknown). If
> you wipe out the usercertificate field from the entry ldap/
> spider01a.iglass.net then that should do it.
This did the trick! I also had to delete userCertificate for dogtagldap/
spider01a.iglass.net and HTTP/spider01a.iglass.net for the other two
certificates not renewing.
>
>
>
>> I was able to clear 4301 error. One ipaCert needed to be updated.
>>
>
> Great!
>
> rob
>
>
>> thanks
>>
>> On Thu, Jun 16, 2016 at 10:22 AM, Rob Crittenden <rcritten at redhat.com
>> <mailto:rcritten at redhat.com>> wrote:
>>
>> Marc Wiatrowski wrote:
>>
>> Thanks Rob,
>>
>> Any suggestions on how make the CA aware of the current serial
>> number?
>>
>>
>> Serial numbers are dolled out like uid numbers, by the 389-ds DNA
>> Plugin. So each CA that has ever issued a certificate has its own
>> range, hence the quite different serial number values.
>>
>> Given that some issued certificates are unknown it stands to reason
>> that replication is broken between one or more masters. Fixing that
>> should resolve (most of) the other issues.
>>
>> Also started seeing the following error from two of the servers,
>> spider01b and spider01o, but not spider01a when to navigate in
>> the web
>> gui. Though it doesn't appear to stop me from doing anything.
>>
>> IPA Error 4301
>> Certificate operation cannot be completed: EXCEPTION (Invalid
>> Crential.)
>>
>>
>> Dogtag does some of its access control by comparing the incoming
>> client certificate with an expected value in its LDAP database, in
>> this case uid=ipara,ou=People,o=ipaca. There you'll find a copy of
>> the client certificate and a description field that contains the
>> expected serial #, subject and issuer.
>>
>> These are out-of-whack if you're getting Invalid Credentials. It
>> could be a number of things so I'd proceed cautiously. Given you
>> have a working master I'd use that as a starting point.
>>
>> Look at the the RA cert is in /etc/httpd/alias:
>>
>> # certutil -L -d /etc/httpd/alias -n ipaCert | grep Serial
>>
>> See if it is the same on all masters, it should be.
>>
>> If it is, look at the uid=ipara entry on all the masters. Again,
>> should be the same.
>>
>> Note that fixing this won't address any replication issues.
>>
>> rob
>>
>>
>> Marc
>>
>> On Tue, Jun 14, 2016 at 2:07 PM, Marc Wiatrowski <wia at iglass.net
>> <mailto:wia at iglass.net>
>> <mailto:wia at iglass.net <mailto:wia at iglass.net>>> wrote:
>>
>>
>>
>> On Tue, Jun 14, 2016 at 11:22 AM, Rob Crittenden
>> <rcritten at redhat.com <mailto:rcritten at redhat.com>
>> <mailto:rcritten at redhat.com <mailto:rcritten at redhat.com>>> wrote:
>>
>> Marc Wiatrowski wrote:
>>
>> Hello, I'm having issues with the 3 ipa
>> certificates of type
>> CA: IPA
>> renewing on 2 of 3 replicas. Particularly on the 2
>> that are
>> not the CA
>> master. The other 5 certificates from getcert list
>> do renew
>> and all
>> certificates on the CA master do look to renew.
>>
>> Both servers running
>> ipa-server-3.0.0-50.el6.centos.1.x86_64 I've done
>> full updates and rebooted.
>>
>>
>> Can you check on the replication status for each CA?
>>
>> $ ipa-csreplica-manage list -v ipa.example.com
>> <http://ipa.example.com>
>> <http://ipa.example.com>
>>
>> The hostname is important because including that will
>> show the
>> agreements that host has. Do this for each master with
>> a CA.
>>
>> The CA being asked to do the renewal is unaware of the
>> current
>> serial number so it is refusing to proceed.
>>
>> rob
>>
>>
>>
>> [root at spider01o]$ ipa-csreplica-manage list -v
>> spider01a.iglass.net <http://spider01a.iglass.net>
>> <http://spider01a.iglass.net>
>> Directory Manager password:
>>
>> spider01b.iglass.net <http://spider01b.iglass.net>
>> <http://spider01b.iglass.net>
>> last init status: None
>> last init ended: None
>> last update status: 0 Replica acquired successfully:
>> Incremental
>> update succeeded
>> last update ended: 2016-06-14 17:49:16+00:00
>> spider01o.iglass.net <http://spider01o.iglass.net>
>> <http://spider01o.iglass.net>
>> last init status: None
>> last init ended: None
>> last update status: 0 Replica acquired successfully:
>> Incremental
>> update started
>> last update ended: 2016-06-14 17:55:20+00:00
>>
>> [root at spider01o]$ ipa-csreplica-manage list -v
>> spider01o.iglass.net <http://spider01o.iglass.net>
>> <http://spider01o.iglass.net>
>> Directory Manager password:
>>
>> spider01a.iglass.net <http://spider01a.iglass.net>
>> <http://spider01a.iglass.net>
>> last init status: None
>> last init ended: None
>> last update status: 0 Replica acquired successfully:
>> Incremental
>> update started
>> last update ended: 2016-06-14 17:57:44+00:00
>> spider01b.iglass.net <http://spider01b.iglass.net>
>> <http://spider01b.iglass.net>
>> last init status: None
>> last init ended: None
>> last update status: 0 Replica acquired successfully:
>> Incremental
>> update started
>> last update ended: 2016-06-14 17:57:41+00:00
>>
>> [root at spider01o]$ ipa-csreplica-manage list -v
>> spider01b.iglass.net <http://spider01b.iglass.net>
>> <http://spider01b.iglass.net>
>> Directory Manager password:
>>
>> spider01a.iglass.net <http://spider01a.iglass.net>
>> <http://spider01a.iglass.net>
>> last init status: 0 Total update succeeded
>> last init ended: 2016-06-03 19:43:12+00:00
>> last update status: 0 Replica acquired successfully:
>> Incremental
>> update succeeded
>> last update ended: 2016-06-14 17:44:17+00:00
>> spider01o.iglass.net <http://spider01o.iglass.net>
>> <http://spider01o.iglass.net>
>> last init status: 0 Total update succeeded
>> last init ended: 2016-06-03 19:44:38+00:00
>> last update status: 0 Replica acquired successfully:
>> Incremental
>> update started
>> last update ended: 2016-06-14 17:57:53+00:00
>> spider01a.iglass.net <http://spider01a.iglass.net>
>> <http://spider01a.iglass.net>
>> last init status: None
>> last init ended: None
>> last update status: 0 Replica acquired successfully:
>> Incremental
>> update succeeded
>> last update ended: 2016-06-14 17:44:13+00:00
>> spider01o.iglass.net <http://spider01o.iglass.net>
>> <http://spider01o.iglass.net>
>> last init status: None
>> last init ended: None
>> last update status: 0 Replica acquired successfully:
>> Incremental
>> update started
>> last update ended: 2016-06-14 17:57:54+00:00
>>
>>
>> Not sure what this is telling... This an issue with the
>> last being
>> doubled? Thanks
>>
>>
>>
>> The failed renews look like:
>>
>> [root at spider01a]$ getcert list -i 20141202144354
>> Number of certificates and requests being tracked: 8.
>> Request ID '20141202144354':
>> status: CA_UNREACHABLE
>> ca-error: Server at https://spider01a.iglass.net/ipa/xml
>> failed request,
>> will retry: 4301 (RPC failed at server. Certificate
>> operation cannot be
>> completed: EXCEPTION (Certificate serial number 0x3ffe0010
>> not found)).
>> stuck: no
>> key pair storage:
>>
>>
>> type=NSSDB,location='/etc/dirsrv/slapd-PKI-IPA',nickname='Server-Cert',token='NSS
>> Certificate
>> DB',pinfile='/etc/dirsrv/slapd-PKI-IPA/pwdfile.txt'
>> certificate:
>>
>>
>> type=NSSDB,location='/etc/dirsrv/slapd-PKI-IPA',nickname='Server-Cert',token='NSS
>> Certificate DB'
>> CA: IPA
>> issuer: CN=Certificate Authority,O=IGLASS.NET
>> <http://IGLASS.NET>
>> <http://iglass.net/> <http://IGLASS.NET <http://iglass.net/
>> >>
>> subject: CN=spider01a.iglass.net
>> <http://spider01a.iglass.net> <http://spider01a.iglass.net/>
>> <http://spider01a.iglass.net
>> <http://spider01a.iglass.net/>>,O=IGLASS.NET
>> <http://IGLASS.NET>
>>
>> <http://iglass.net/> <http://IGLASS.NET <http://iglass.net/
>> >>
>> expires: 2016-12-02 14:38:45 UTC
>> key usage:
>>
>> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>> eku: id-kp-serverAuth,id-kp-clientAuth
>> pre-save command:
>> post-save command: /usr/lib64/ipa/certmonger/restart_dirsrv
>> PKI-IPA
>> track: yes
>> auto-renew: yes
>>
>> [root at spider01a]$ getcert list -i 20141202144616
>> Number of certificates and requests being tracked: 8.
>> Request ID '20141202144616':
>> status: CA_UNREACHABLE
>> ca-error: Server at https://spider01a.iglass.net/ipa/xml
>> failed request,
>> will retry: 4301 (RPC failed at server. Certificate
>> operation cannot be
>> completed: EXCEPTION (Certificate serial number 0x3ffe000f
>> not found)).
>> stuck: no
>> key pair storage:
>>
>>
>> type=NSSDB,location='/etc/dirsrv/slapd-IGLASS-NET',nickname='Server-Cert',token='NSS
>> Certificate
>> DB',pinfile='/etc/dirsrv/slapd-IGLASS-NET/pwdfile.txt'
>> certificate:
>>
>>
>> type=NSSDB,location='/etc/dirsrv/slapd-IGLASS-NET',nickname='Server-Cert',token='NSS
>> Certificate DB'
>> CA: IPA
>> issuer: CN=Certificate Authority,O=IGLASS.NET
>> <http://IGLASS.NET>
>> <http://iglass.net/> <http://IGLASS.NET <http://iglass.net/
>> >>
>> subject: CN=spider01a.iglass.net
>> <http://spider01a.iglass.net> <http://spider01a.iglass.net/>
>> <http://spider01a.iglass.net
>> <http://spider01a.iglass.net/>>,O=IGLASS.NET
>> <http://IGLASS.NET>
>>
>> <http://iglass.net/> <http://IGLASS.NET <http://iglass.net/
>> >>
>> expires: 2016-12-02 14:38:43 UTC
>> key usage:
>>
>> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>> eku: id-kp-serverAuth,id-kp-clientAuth
>> pre-save command:
>> post-save command: /usr/lib64/ipa/certmonger/restart_dirsrv
>> IGLASS-NET
>> track: yes
>> auto-renew: yes
>>
>> [root at spider01a]$ getcert list -i 20141202144733
>> Number of certificates and requests being tracked: 8.
>> Request ID '20141202144733':
>> status: CA_UNREACHABLE
>> ca-error: Server at https://spider01a.iglass.net/ipa/xml
>> failed request,
>> will retry: 4301 (RPC failed at server. Certificate
>> operation cannot be
>> completed: EXCEPTION (Certificate serial number 0x3ffe0011
>> not found)).
>> stuck: no
>> key pair storage:
>>
>>
>> type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
>> Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
>> certificate:
>>
>>
>> type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
>> Certificate DB'
>> CA: IPA
>> issuer: CN=Certificate Authority,O=IGLASS.NET
>> <http://IGLASS.NET>
>> <http://iglass.net/> <http://IGLASS.NET <http://iglass.net/
>> >>
>> subject: CN=spider01a.iglass.net
>> <http://spider01a.iglass.net> <http://spider01a.iglass.net/>
>> <http://spider01a.iglass.net
>> <http://spider01a.iglass.net/>>,O=IGLASS.NET
>> <http://IGLASS.NET>
>>
>> <http://iglass.net/> <http://IGLASS.NET <http://iglass.net/
>> >>
>> expires: 2016-12-02 14:38:46 UTC
>> key usage:
>>
>> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>> eku: id-kp-serverAuth,id-kp-clientAuth
>> pre-save command:
>> post-save command: /usr/lib64/ipa/certmonger/restart_httpd
>> track: yes
>> auto-renew: yes
>>
>>
>> From
>> [root at spider01a]$ getcert resubmit -i 20141202144354
>>
>> On the replica issuing the resubmit
>>
>> ==> /var/log/httpd/access_log <==
>> 192.168.176.2 - - [13/Jun/2016:15:49:32 -0400] "POST
>> /ipa/xml HTTP/1.1"
>> 401 1370
>>
>> ==> /var/log/httpd/error_log <==
>> [Mon Jun 13 15:49:33 2016] [error] ipa: ERROR:
>> ipaserver.plugins.dogtag.ra.get_certificate(): EXCEPTION
>> (Certificate
>> serial number 0x3ffe0010 not found)
>> [Mon Jun 13 15:49:33 2016] [error] ipa: INFO:
>> host/spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>>>:
>>
>>
>> cert_request(u'MIIDsTCCApkCAQAwNDETMBEGA1UEChMKSUdMQVNTLk5FVDEdMBsGA1UEAxMUc3BpZGVyMDFhLml...UVrN8lbKn17V5COjnj6k0mdbz3KptL0UI/l0BPlFBWGN5MFYaDx2F+y6LWv/aXeu2V4E6LA==',
>> principal=u'dogtagldap/spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>>>', add=True):
>> CertificateOperationError
>>
>> ==> /var/log/httpd/access_log <==
>> 192.168.176.2 - - [13/Jun/2016:15:49:33 -0400] "POST
>> /ca/agent/ca/displayBySerial HTTP/1.1" 200 262
>> 192.168.176.2 - host/spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>>> [13/Jun/2016:15:49:32
>> -0400]
>> "POST /ipa/xml HTTP/1.1" 200 376
>>
>> ==> /var/log/pki-ca/system <==
>> 2508.TP-Processor6 - [13/Jun/2016:15:49:33 EDT] [3] [3]
>> Servlet
>> caDisplayBySerial: Error encountered in DisplayBySerial.
>> Error Record
>> not found.
>>
>>
>> On the CA master spider01o:
>>
>> ==> /var/log/httpd/access_log <==
>> 192.168.176.2 - - [13/Jun/2016:15:49:33 -0400] "POST
>> /ipa/xml HTTP/1.1"
>> 401 1370
>>
>> ==> krb5kdc.log <==
>> Jun 13 15:49:34 spider01o.iglass.net
>> <http://spider01o.iglass.net>
>> <http://spider01o.iglass.net/> <http://spider01o.iglass.net
>> <http://spider01o.iglass.net/>>
>> krb5kdc[1963](info): TGS_REQ (4 etypes {18 17 16 23})
>> 192.168.177.2
>> <http://192.168.177.2 <http://192.168.177.2/>>: ISSUE:
>> authtime
>> 1465847372, etypes {rep=18
>> tkt=18 ses=18}, host/spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>>> for
>> ldap/spider01o.iglass.net at IGLASS.NET
>> <mailto:spider01o.iglass.net at IGLASS.NET>
>> <mailto:spider01o.iglass.net at IGLASS.NET
>> <mailto:spider01o.iglass.net at IGLASS.NET>>
>> <mailto:spider01o.iglass.net at IGLASS.NET
>> <mailto:spider01o.iglass.net at IGLASS.NET>
>> <mailto:spider01o.iglass.net at IGLASS.NET
>> <mailto:spider01o.iglass.net at IGLASS.NET>>>
>>
>> ==> /var/log/httpd/error_log <==
>> [Mon Jun 13 15:49:34 2016] [error] ipa: ERROR:
>> ipaserver.plugins.dogtag.ra.get_certificate(): EXCEPTION
>> (Invalid
>> Credential.)
>> [Mon Jun 13 15:49:34 2016] [error] ipa: INFO:
>> host/spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>>>:
>>
>>
>> cert_request(u'MIIDsTCCApkCAQAwNDETMBEGA1UEChMKSUdMQVNTLk5FVDEdMBsGA1UEAxMUc3BpZGVyMDFhLml...UVrN8lbKn17V5COjnj6k0mdbz3KptL0UI/l0BPlFBWGN5MFYaDx2F+y6LWv/aXeu2V4E6LA==',
>> principal=u'dogtagldap/spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>>>', add=True):
>> CertificateOperationError
>>
>> ==> /var/log/httpd/access_log <==
>> 192.168.177.2 - - [13/Jun/2016:15:49:34 -0400] "POST
>> /ca/agent/ca/displayBySerial HTTP/1.1" 200 235
>> 192.168.176.2 - host/spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>
>> <mailto:spider01a.iglass.net at IGLASS.NET
>> <mailto:spider01a.iglass.net at IGLASS.NET>>> [13/Jun/2016:15:49:33
>> -0400]
>> "POST /ipa/xml HTTP/1.1" 200 349
>>
>> ==> /var/log/pki-ca/system <==
>> 2231.TP-Processor3 - [13/Jun/2016:15:49:34 EDT] [6] [3]
>> Cannot
>> authenticate agent with certificate Serial 0x5ffc0008
>> Subject DN CN=IPA
>> RA,O=IGLASS.NET <http://IGLASS.NET> <http://iglass.net/>
>> <http://IGLASS.NET
>> <http://iglass.net/>>. Error: User not found
>>
>>
>> I realize they expire at the end of the year, but I've had my
>> certificates expire before and would rather not go through
>> that again.
>> Any idea on what's wrong or suggestions on where to look
>> would be
>> appreciated.
>>
>> Thanks,
>> Marc
>>
>>
>>
>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/freeipa-users/attachments/20160622/730c02f2/attachment.htm>
More information about the Freeipa-users
mailing list