[Freeipa-users] IPA server having cert issues
Bret Wortman
bret.wortman at damascusgrp.com
Wed Apr 27 14:46:00 UTC 2016
So in lieu of fixing these certs, is there an acceptable way to dump
them all and start over /without losing the contents of the IPA
database/? Or otherwise really screwing ourselves?
We have a replica that's still up and running and we've switched
everyone over to talking to it, but we're at risk with just the one.
Thanks!
On 04/27/2016 06:05 AM, Bret Wortman wrote:
> Was this at all informative?
>
> On 04/26/2016 02:06 PM, Bret Wortman wrote:
>>
>>
>> On 04/26/2016 01:45 PM, Rob Crittenden wrote:
>>> Bret Wortman wrote:
>>>> I think I've found a deeper problem, in that I can't update these
>>>> because IPA simply won't start at all now.
>>>>
>>>> I mistyped one of these -- the 2016-03-11 is actually 2018-03-11, and
>>>> 2016-04-01 is actually 2036-04-01.
>>>>
>>>> As for the unknowns, the first says status: CA_REJECTED and the error
>>>> says "hostname in subject of request 'zw198.private.net' does not
>>>> match
>>>> principal hostname 'private.net'", with stuck: yes.
>>>>
>>>> The second is similar, but for a different host.
>>>
>>> Is it really a different host and why? I think we'd need to see the
>>> full output to know what's going on.
>>>
>>
>> Full output:
>>
>> Number of certificates and requests being tracked: 10.
>> Request ID '20140428181940':
>> status: MONITORING
>> stuck: no
>> key pair storage:
>> type=NSSDB,location='/etc/dirsrv/slapd-PRIVATE-NET',nickname='Server-Cert',token='NSS
>> Certificate DB',pinfile='/etc/dirsrv/slapd-PRIVATE-NET/pwdfile.txt'
>> certificate:
>> type=NSSDB,location='/etc/dirsrv/slapd-PRIVATE-NET',nickname='Server-Cert',token='NSS
>> Certificate DB'
>> CA: IPA
>> issuer: CN=Certificate Authority,O=PRIVATE.NET
>> subject: CN=zsipa.private.net,O=PRIVATE.NET
>> expires: 2018-04-02 13:04:51 UTC
>> principal name: ldap/zsipa.private.net at PRIVATE.NET
>> key usage:
>> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>> eku: id-kp-serverAuth,id-kp-clientAuth
>> pre-save command:
>> post-save command:
>> track: yes
>> auto-renew: yes
>> Request ID '20140428182016':
>> status: MONITORING
>> stuck: no
>> key pair storage:
>> type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
>> Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
>> certificate:
>> type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
>> Certificate DB'
>> CA: IPA
>> issuer: CN=Certificate Authority,O=PRIVATE.NET
>> subject: CN=zsipa.private.net,O=PRIVATE.NET
>> expires: 2018-04-02 13:04:31 UTC
>> principal name: HTTP/zsipa.private.net at PRIVATE.NET
>> key usage:
>> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>> eku: id-kp-serverAuth,id-kp-clientAuth
>> pre-save command:
>> post-save command:
>> track: yes
>> auto-renew: yes
>> Request ID '20150211141945':
>> status: CA_REJECTED
>> ca-error: Server at https://zsipa.private.net/ipa/xml denied our
>> request, giving up: 2100 (RPC failed at server. Insufficient access:
>> hostname in subject of request 'zw198.private.net' does not match
>> principal hostname 'private.net').
>> stuck: yes
>> key pair storage:
>> type=NSSDB,location='/etc/pki/nssdb',nickname='Server-Cert',token='NSS Certificate
>> DB'
>> certificate:
>> type=NSSDB,location='/etc/pki/nssdb',nickname='Server-Cert'
>> CA: IPA
>> issuer:
>> subject:
>> expires: unknown
>> pre-save command:
>> post-save command:
>> track: yes
>> auto-renew: yes
>> Request ID '20150816194107':
>> status: CA_UNREACHABLE
>> ca-error: Internal error
>> stuck: no
>> key pair storage:
>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
>> cert-pki-ca',token='NSS Certificate DB',pin='424151811070'
>> certificate:
>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
>> cert-pki-ca',token='NSS Certificate DB'
>> CA: dogtag-ipa-ca-renew-agent
>> issuer: CN=Certificate Authority,O=PRIVATE.NET
>> subject: CN=CA Audit,O=PRIVATE.NET
>> expires: 2016-04-17 18:19:19 UTC
>> key usage: digitalSignature,nonRepudiation
>> pre-save command:
>> post-save command:
>> track: yes
>> auto-renew: yes
>> Request ID '20150816194108':
>> status: CA_UNREACHABLE
>> ca-error: Internal error
>> stuck: no
>> key pair storage:
>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
>> cert-pki-ca',token='NSS Certificate DB',pin='424151811070'
>> certificate:
>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
>> cert-pki-ca',token='NSS Certificate DB'
>> CA: dogtag-ipa-ca-renew-agent
>> issuer: CN=Certificate Authority,O=PRIVATE.NET
>> subject: CN=OCSP Subsystem,O=PRIVATE.NET
>> expires: 2016-04-17 18:19:18 UTC
>> key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
>> eku: id-kp-OCSPSigning
>> pre-save command:
>> post-save command:
>> track: yes
>> auto-renew: yes
>> Request ID '20150816194109':
>> status: CA_UNREACHABLE
>> ca-error: Internal error
>> stuck: no
>> key pair storage:
>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
>> cert-pki-ca',token='NSS Certificate DB',pin='424151811070'
>> certificate:
>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
>> cert-pki-ca',token='NSS Certificate DB'
>> CA: dogtag-ipa-ca-renew-agent
>> issuer: CN=Certificate Authority,O=PRIVATE.NET
>> subject: CN=CA Subsystem,O=PRIVATE.NET
>> expires: 2016-04-17 18:19:19 UTC
>> key usage:
>> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>> eku: id-kp-serverAuth,id-kp-clientAuth
>> pre-save command:
>> post-save command:
>> track: yes
>> auto-renew: yes
>> Request ID '20150816194110':
>> status: MONITORING
>> stuck: no
>> key pair storage:
>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
>> cert-pki-ca',token='NSS Certificate DB',pin='424151811070'
>> certificate:
>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
>> cert-pki-ca',token='NSS Certificate DB'
>> CA: dogtag-ipa-ca-renew-agent
>> issuer: CN=Certificate Authority,O=PRIVATE.NET
>> subject: CN=Certificate Authority,O=PRIVATE.NET
>> expires: 2036-04-01 20:16:39 UTC
>> key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
>> pre-save command:
>> post-save command:
>> track: yes
>> auto-renew: yes
>> Request ID '20150816194111':
>> status: CA_UNREACHABLE
>> ca-error: Internal error
>> stuck: no
>> key pair storage:
>> type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
>> Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
>> certificate:
>> type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
>> Certificate DB'
>> CA: dogtag-ipa-ca-renew-agent
>> issuer: CN=Certificate Authority,O=PRIVATE.NET
>> subject: CN=IPA RA,O=PRIVATE.NET
>> expires: 2016-04-17 18:19:35 UTC
>> key usage:
>> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>> eku: id-kp-serverAuth,id-kp-clientAuth
>> pre-save command:
>> post-save command:
>> track: yes
>> auto-renew: yes
>> Request ID '20150816194112':
>> status: MONITORING
>> stuck: no
>> key pair storage:
>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
>> cert-pki-ca',token='NSS Certificate DB',pin='424151811070'
>> certificate:
>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
>> cert-pki-ca',token='NSS Certificate DB'
>> CA: dogtag-ipa-renew-agent
>> issuer: CN=Certificate Authority,O=PRIVATE.NET
>> subject: CN=zsipa.private.net,O=PRIVATE.NET
>> expires: 2018-03-11 13:04:29 UTC
>> key usage:
>> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>> eku: id-kp-serverAuth,id-kp-clientAuth
>> pre-save command:
>> post-save command:
>> track: yes
>> auto-renew: yes
>> Request ID '20151214165433':
>> status: CA_REJECTED
>> ca-error: Server at https://zsipa.private.net/ipa/xml denied our
>> request, giving up: 2100 (RPC failed at server. Insufficient access:
>> hostname in subject of request 'zsipa.private.net' does not match
>> principal hostname 'www.private.net').
>> stuck: yes
>> key pair storage:
>> type=FILE,location='/etc/pki/tls/private/www.private.net.key'
>> certificate:
>> type=FILE,location='/etc/pki/tls/certs/www.private.net.crt'
>> CA: IPA
>> issuer:
>> subject:
>> expires: unknown
>> pre-save command:
>> post-save command:
>> track: yes
>> auto-renew: yes
>>
>>
>>> A given host can only get certificates for itself or those delegated
>>> to it. Hostnames are used for this enforcement so if they don't line
>>> up you'll see this type of rejection.
>>>
>>>>
>>>> No idea what's wrong with the rest, or why nothing will start. Near
>>>> as I
>>>> can tell, Kerberos is failing to start, which is causing everything
>>>> else
>>>> to go toes up.
>>>>
>>>> Early in the startup, in /var/log/messages, there's:
>>>>
>>>> ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code may
>>>> provide
>>>> more information (No Kerberos credentials available)
>>>
>>> Without more context it's hard to say. 389 is rather chatty about
>>> things and of course when it starts it has no ticket so it logs a
>>> bunch of stuff, eventually (hopefully) gets one, and then shuts up.
>>>
>>>>
>>>> After that, I get a jar file read pboelm on log4j.jar, then a
>>>> series of
>>>> property setting attempts that don't find matching properties. Then
>>>> some
>>>> cipher errors, then it looks like named starts up okay, and everything
>>>> pauses for about 5 minutes before it all comes crashing back down.
>>>>
>>>
>>> I wouldn't get too hung up on particular services just yet. Without
>>> valid certs things will fail and those problems will cascade. I
>>> think we just need more details at this point.
>>>
>>> rob
>>>
>>>>
>>>> Bret
>>>>
>>>> On 04/26/2016 12:40 PM, Petr Vobornik wrote:
>>>>> On 04/26/2016 06:00 PM, Bret Wortman wrote:
>>>>>> # getcert list | grep expires
>>>>>> expires: 2018-04-02 13:04:51 UTC
>>>>>> expires: 2018-04-02 13:04:31 UTC
>>>>>> expires: unknown
>>>>>> expires: 2016-04-17 18:19:19 UTC
>>>>>> expires: 2016-04-17 18:19:18 UTC
>>>>>> expires: 2016-04-17 18:19:19 UTC
>>>>>> expires: 2016-04-01 20:16:39 UTC
>>>>>> expires: 2016-04-17 18:19:35 UTC
>>>>>> expires: 2016-03-11 13:04:29 UTC
>>>>>> expires: unknown
>>>>>> #
>>>>>>
>>>>>> So some got updated and most didn't. Is there a recommended way
>>>>>> to update these
>>>>>> all? The system is still backdated to 3 April (ntpd disabled) at
>>>>>> this point.
>>>>> It's usually good to start renewing(when it doesn't happen
>>>>> automatically
>>>>> from some reason) with the cert which is about to expired first, i.e.
>>>>> the one with "2016-03-11 13:04:29"
>>>>>
>>>>> The process is:
>>>>> - move date before the cert is about to expired
>>>>> - leave it up to certmonger or manually force resubmit by `getcert
>>>>> resubmit -i $REQUEST_ID`, where request ID is in `getcert list`
>>>>> output.
>>>>>
>>>>> I'm little worried about the fact that CA cert was renewed at date
>>>>> which
>>>>> is after expiration of the other certs.
>>>>>
>>>>> Also the `expires: unknown` doesn't look good. Check `getcert list`
>>>>> output for errors related to the cert.
>>>>>
>>>>>
>>>>>>
>>>>>> Bret
>>>>>>
>>>>>>
>>>>>> On 04/26/2016 11:46 AM, Petr Vobornik wrote:
>>>>>>> On 04/26/2016 03:26 PM, Bret Wortman wrote:
>>>>>>>> On our non-CA IPA server, this is happening, in case it's
>>>>>>>> related and illustrative:
>>>>>>>>
>>>>>>>> # ipa host-del zw113.private.net
>>>>>>>> ipa: ERROR: Certificate format error:
>>>>>>>> (SEC_ERROR_LEGACY_DATABASE) The
>>>>>>>> certificate/key database is in an old, unsupported format.
>>>>>>>> #
>>>>>>> I would start with checking on all IPA servers if and what
>>>>>>> certificates
>>>>>>> are expired:
>>>>>>> # getcert list
>>>>>>> or short version to check if there are any:
>>>>>>> # getcert list | grep expires
>>>>>>>
>>>>>>> When CA cert is renewed, it is not automatically transfered to
>>>>>>> clients.
>>>>>>> There one must run:
>>>>>>> # ipa-certupdate
>>>>>>>
>>>>>>>> On 04/26/2016 09:24 AM, Bret Wortman wrote:
>>>>>>>>> I rolled the date on the IPA server in question back to April
>>>>>>>>> 1 and ran
>>>>>>>>> "ipa-cacert-manage renew", which said it completed
>>>>>>>>> successfully. I rolled the
>>>>>>>>> date back to current and tried restarting ipa using ipactl
>>>>>>>>> stop && ipactl
>>>>>>>>> start, but no joy. No more ca renewal errors, but right after
>>>>>>>>> the pause I see
>>>>>>>>> this in /var/log/messages:
>>>>>>>>>
>>>>>>>>> systemd: kadmin.service: main process exited, code=exited,
>>>>>>>>> status=2/INVALIDARGUMENT
>>>>>>>>> systemd: Unit kadmin.service entered failed state.
>>>>>>>>> systemd: kadmin.service failed.
>>>>>>>>>
>>>>>>>>> I rebooted the server just in case, and it's still getting
>>>>>>>>> stuck at the same
>>>>>>>>> place. ipa-otpd doesn't get around to starting.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Bret
>>>>>>>>>
>>>>>>>>> After the several-minutes-long pause after ipactl start
>>>>>>>>> outputs "Starting
>>>>>>>>> pki-tomcatd Service", I get the
>>>>>>>>>
>>>>>>>>> On 04/26/2016 08:14 AM, Bret Wortman wrote:
>>>>>>>>>> I have an IPA server on a private network which has
>>>>>>>>>> apparently run into
>>>>>>>>>> certificate issues this morning. It's been running without
>>>>>>>>>> issue for quite a
>>>>>>>>>> while, and is on 4.1.4-1 on fedora 21.
>>>>>>>>>>
>>>>>>>>>> This morning, the gui started giving:
>>>>>>>>>>
>>>>>>>>>> IPA Error 907: NetworkError with description "cannot connect to
>>>>>>>>>> 'https://zsipa.private.net:443/ca/agent/ca/displayBySerial':
>>>>>>>>>> (SSL_ERROR_EXPIRED_CERRT_ALERT) SSL peer rejected your
>>>>>>>>>> certificate as expired."
>>>>>>>>>>
>>>>>>>>>> I dug into the logs and after trying to restart ipa using
>>>>>>>>>> ipactl, there was a
>>>>>>>>>> length pause, then:
>>>>>>>>>>
>>>>>>>>>> dogtag-ipa-ca-renew-agent-submit: Updated certificate not
>>>>>>>>>> available
>>>>>>>>>> certmonger: Certificate named "ipaCert" in token "NSS
>>>>>>>>>> Certificate DB" in
>>>>>>>>>> database "/etc/httpd/alias" is no longer valid.
>>>>>>>>>> dogtag-ipa-ca-renew-agent-submit: Updated certificate not
>>>>>>>>>> available
>>>>>>>>>> certmonger: Certificate named "ocspSigningCert cert-pki-ca"
>>>>>>>>>> in token "NSS
>>>>>>>>>> Certificate DB" in database "/etc/pki/pki-tomcat/alias" is no
>>>>>>>>>> longer valid.
>>>>>>>>>> dogtag-ipa-ca-renew-agent-submit: Updated certificate not
>>>>>>>>>> available.
>>>>>>>>>> named-pkcs11[3437]: client 192.168.208.205#57832: update
>>>>>>>>>> '208.168.192.in-addr.arpa/IN' denied
>>>>>>>>>>
>>>>>>>>>> and then things start shutting down. I can't start ipa at all
>>>>>>>>>> using ipactl.
>>>>>>>>>>
>>>>>>>>>> So at present, our DNS is down. Authentication should work
>>>>>>>>>> for a while, but
>>>>>>>>>> I'd like to get this working again as quickly as possible.
>>>>>>>>>> Any ideas? I deal
>>>>>>>>>> with certificates so infrequently (like only when something
>>>>>>>>>> like this
>>>>>>>>>> happens) that I'm not sure where to start.
>>>>>>>>>>
>>>>>>>>>> Thanks!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Bret Wortman*
>>>>>>>>>> /Coming soon to Kickstarter.../
>>>>>>>>>> <http://wrapbuddies.co/>
>>>>>>>>>> http://wrapbuddies.co/
>>>>>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/freeipa-users/attachments/20160427/a6e8168b/attachment.htm>
More information about the Freeipa-users
mailing list