[Freeipa-users] CA expiration and renewal

Thu Dec 5 20:39:47 UTC 2013

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/05/2013 12:18 PM, Rob Crittenden wrote:
> Erinn Looney-Triggs wrote:
>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
>> 
>> On 12/05/2013 01:35 AM, Martin Kosek wrote:
>>> On 12/04/2013 06:58 PM, Erinn Looney-Triggs wrote:
>>>> On 12/04/2013 07:15 AM, Rob Crittenden wrote:
>>>>> Erinn Looney-Triggs wrote:
>>>>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
>>>>>> 
>>>>>> On 11/27/2013 11:11 AM, Rob Crittenden wrote:
>>>>>>> Erinn Looney-Triggs wrote:
>>>>>>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On 11/25/2013 11:09 AM, Rob Crittenden wrote:
>>>>>>>>> Erinn Looney-Triggs wrote:
>>>>>>>>>> Folks just wanted to touch base again before the 
>>>>>>>>>> American holiday season starts. My CA, which is 
>>>>>>>>>> subordinate to AD CS will be expiring on
>>>>>>>>>> December 9th, I submitted a bug, y'all drew up
>>>>>>>>>> docs etc for a plan (thanks). Now I just wanted
>>>>>>>>>> to see how it was going and if need be what
>>>>>>>>>> manual steps I will need to take to renew the
>>>>>>>>>> certificate.
>>>>>>>>>> 
>>>>>>>>>> Thanks again for the great work,
>>>>>>>>> 
>>>>>>>>> We're working on an a set of tools to make this
>>>>>>>>> easier. For now I've appended some manual
>>>>>>>>> instructions onto a page still in progress.
>>>>>>>>> 
>>>>>>>>> http://www.freeipa.org/page/Howto/CA_Certificate_Renewal#Manual_Procedure_in_IPA_3.0
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>
>>>>>>>>>
>>>>
>>>>>>>>>
>>>
>>>>>>>>>
>>
>>>>>>>>> 
Some parts may be still be a little rough or hard to understand.
>>>>>>>>> Let me know if you have any problems or
>>>>>>>>> corrections.
>>>>>>>>> 
>>>>>>>>> rob
>>>>>>>> 
>>>>>>>> Rob,
>>>>>>>> 
>>>>>>>> Thanks for the instructions, a few questions.
>>>>>>>> 
>>>>>>>> What sort of interruption in service could this
>>>>>>>> create?
>>>>>>> 
>>>>>>> Services will be restarted during this process
>>>>>>> including your LDAP, Apache and CA instances. Downtime
>>>>>>> should be relatively short, no more than a few minutes
>>>>>>> combined.
>>>>>>> 
>>>>>>>> Can you expand on this section a little bit: Replace
>>>>>>>> the value of ca.signing.cert in /etc/pki-ca/CS.cfg.
>>>>>>>> This is the base64 value of the certificate. You can
>>>>>>>> obtain this by removing the BEGIN/END blocks from
>>>>>>>> ipa.crt and compressing it into a single line.
>>>>>>> 
>>>>>>> A PEM cert looks like:
>>>>>>> 
>>>>>>> -----BEGIN CERTIFICATE----- 
>>>>>>> MIIB/zCCAWigAwIBAgICA+gwDQYJKoZIhvcNAQEFBQAwKTEnMCUGA1UEAxMeSVBB
>>>>>>>
>>>>>>>
>>>>
>>>>>>>
>>
>>>>>>> 
IFRlc3QgQ2VydGlmaWNhdGUgQXV0aG9yaXR5MB4XDTEwMDIyMzIyMzMxNVoXDTIw
>>>>>>> MDIyMzIyMzMxNVowKTEnMCUGA1UEAxMeSVBBIFRlc3QgQ2VydGlmaWNhdGUgQXV0
>>>>>>>
>>>>>>>
>>>>
>>>>>>>
>>
>>>>>>> 
aG9yaXR5MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC+G6ultyLaXqzBlypA
>>>>>>> DnOsinkMTlZZssTFQh/QUMi1F1fcn8QUlmsl9a+l6w6hfZm7P8z3sVwsjLQcDWA4
>>>>>>>
>>>>>>>
>>>>
>>>>>>>
>>
>>>>>>> 
KxOh+LmIsNL4OKx4wKF1q/pSt1PATRU5Pgu2+3wlwJO0H7cl4QfavoOLwmxAZf/l
>>>>>>> ZNIy/5czvSWFWj7EJj16ty9BeQIDAQABozYwNDARBglghkgBhvhCAQEEBAMCAAcw
>>>>>>>
>>>>>>>
>>>>
>>>>>>>
>>
>>>>>>> 
DwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAsQwDQYJKoZIhvcNAQEFBQAD
>>>>>>> gYEAl0gIshwNkhyfNe1XMLswPeOgH5YN1BUuKXzbv1fuSIkArjwLODr4cOdXzQvt
>>>>>>>
>>>>>>>
>>>>
>>>>>>>
>>
>>>>>>> 
yaiX6Z+pRC/sK8MgLhPxV2X9QVQdzKfLkVGIdboCt1j3EXxSUCZeIuSKouitkWYe
>>>>>>> eSH9DQkYDp/oKgANLWnY7CNorPz6xQktp1pB0DGohV1BeTA=
>>>>>>> -----END CERTIFICATE-----
>>>>>>> 
>>>>>>> You need to drop the BEGIN/END blocks then combine all
>>>>>>> the lines into a single line, so you have a unified
>>>>>>> base64 blog. It will look like:
>>>>>>> 
>>>>>>> ca.signing.cert=MII...B0DGohV1BeTA=
>>>>>>> 
>>>>>>> I was afraid wrapping woudl destroy my demonstration so
>>>>>>> I used ellipses instead.
>>>>>>> 
>>>>>>>> Thanks and happy Thanksgiving,
>>>>>>> 
>>>>>>> You're welcome. You too.
>>>>>>> 
>>>>>>> rob
>>>>>>> 
>>>>>> 
>>>>>> Ok I have done the steps as outlined. One small
>>>>>> suggestion and one question came up.
>>>>>> 
>>>>>> Suggestion: for the ldapmodify command indicate that a
>>>>>> ctl-d is necessary to end input. Most folks will know
>>>>>> this, but some may not.
>>>>>> 
>>>>>> For the client section you have me copy the newly signed 
>>>>>> subordnate CA certificate into /etc/ipa/ca.crt. However,
>>>>>> on my hosts that was actually a copy of the AD CS
>>>>>> certificate, not the subordinate certificate. In the case
>>>>>> of a subordinate installation do you want the root or the
>>>>>> subordinate CA? It would seem that the root would be
>>>>>> broader, but I just want to make sure.
>>>>>> 
>>>> 
>>>>> The IPA CA cert should be sufficient.
>>>> 
>>>>> rob
>>>> 
>>>> 
>>>> Thanks, and just for an update, the switch over was made, 
>>>> certmonger is happily updating certs now on all hosts and 
>>>> everything just appears to be working thus far, minus the 
>>>> replication of the agent certificate which I am still
>>>> looking into.
>>>> 
>>>> Thanks for the help,
>>>> 
>>>> -Erinn
>>> 
>>> Great, I am glad to hear that. Note that we were investigating 
>>> renewing certificates and clones and found out an issue in
>>> Python readline that prevented a renewal of the IPA agent
>>> certificate:
>>> 
>>> https://fedorahosted.org/freeipa/ticket/4064
>>> 
>>> Could this be the reason of your issues? Did you saw a crash
>>> of certmonger during the renewal? It was found out to be
>>> happening due to the aforementioned bug.
>>> 
>>> Thanks, Martin
>>> 
>> 
>> That seems very likely, however abrt didn't catch anything, and
>> there doesn't appear to be any tmp file wreckage left anywhere. I
>> can't find anything in the logs indicating failure, all signs
>> point to success for the renewal:
>> 
>> 
>> Dec  3 20:47:25 ipa2 certmonger: Certificate named "Server-Cert"
>> in token "NSS Certificate DB" in database
>> "/etc/dirsrv/slapd-ABAQIS-COM" will not be valid afte r
>> 20131210032326. Dec  3 20:47:25 ipa2 certmonger: Certificate
>> named "Server-Cert" in token "NSS Certificate DB" in database
>> "/etc/dirsrv/slapd-PKI-IPA" will not be valid after
>> 20131210032326. Dec  3 20:47:25 ipa2 certmonger: Certificate
>> named "auditSigningCert cert-pki-ca" in token "NSS Certificate
>> DB" in database "/var/lib/pki-ca/alias" will not be valid after
>> 20131210032326. Dec  3 20:47:25 ipa2 certmonger: Certificate
>> named "ocspSigningCert cert-pki-ca" in token "NSS Certificate DB"
>> in database "/var/lib/pki-ca/alias" will not be valid after
>> 20131210032326. Dec  3 20:47:25 ipa2 certmonger: Certificate
>> named "subsystemCert cert-pki-ca" in token "NSS Certificate DB"
>> in database "/var/lib/pki-ca/alias" will not be valid after
>> 20131210032326. Dec  3 20:47:25 ipa2 certmonger: Certificate
>> named "ipaCert" in token "NSS Certificate DB" in database
>> "/etc/httpd/alias" will not be valid after 20131210032326. Dec  3
>> 20:47:26 ipa2 certmonger: Certificate named "Server-Cert" in 
>> token "NSS Certificate DB" in database
>> "/etc/dirsrv/slapd-PKI-IPA" issued by CA and saved. Dec  3
>> 20:47:26 ipa2 certmonger: Certificate named "Server-Cert" in 
>> token "NSS Certificate DB" in database
>> "/etc/dirsrv/slapd-ABAQIS-COM" issued by CA and saved. Dec  3
>> 20:47:26 ipa2 python: Updating certificate for auditSigningCert 
>> cert-pki-ca Dec  3 20:47:26 ipa2 python: Updating certificate for
>> ocspSigningCert cert-pki-ca Dec  3 20:47:27 ipa2 python: Updating
>> certificate for subsystemCert cert-pki-ca Dec  3 20:47:27 ipa2
>> python: Updating certificate for ipaCert Dec  3 20:47:28 ipa2
>> python: certmonger stopping pki-cad Dec  3 20:48:04 ipa2 python:
>> certmonger started pki-cad, nickname 'auditSigningCert
>> cert-pki-ca' Dec  3 20:48:04 ipa2 certmonger: Certificate named
>> "auditSigningCert cert-pki-ca" in token "NSS Certificate DB" in
>> database "/var/lib/pki-ca/alias" issued by CA and saved. Dec  3
>> 20:48:08 ipa2 python: certmonger stopping pki-cad Dec  3 20:48:44
>> ipa2 python: certmonger started pki-cad, nickname 
>> 'ocspSigningCert cert-pki-ca' Dec  3 20:48:44 ipa2 certmonger:
>> Certificate named "ocspSigningCert cert-pki-ca" in token "NSS
>> Certificate DB" in database "/var/lib/pki-ca/alias" issued by CA
>> and saved. Dec  3 20:48:48 ipa2 python: certmonger stopping
>> pki-cad Dec  3 20:49:24 ipa2 python: certmonger started pki-cad,
>> nickname 'subsystemCert cert-pki-ca' Dec  3 20:49:24 ipa2
>> certmonger: Certificate named "subsystemCert cert-pki-ca" in
>> token "NSS Certificate DB" in database "/var/lib/pki-ca/alias"
>> issued by CA and saved. Dec  3 20:49:27 ipa2 python: certmonger
>> restarted httpd Dec  3 20:49:29 ipa2 certmonger: Certificate
>> named "ipaCert" in token "NSS Certificate DB" in database
>> "/etc/httpd/alias" issued by CA and saved.
>> 
>> 
>> Sorry for the word wrap there. Certmonger continued to run
>> throughout it appears. The dates line up correctly, certmonger on
>> the primary renewed on the 3rd and the secondary failed to get
>> the new certificate which led straight back to the same place.
> 
> But you were able to fix it again, right?
> 
> I wonder if there are any AVCs around renewal time.
> 
> rob
> 

Yep manually moving over and importing worked just fine, as is
everything is good for the next two years until the whole cycle
repeats. But I reckon this would be a very good thing to get sussed out.

Yeah there are AVC messages:

node=ipa2.abaqis.com type=PATH msg=audit(1386103646.841:451293):
item=0 name="/var/run/certmonger/tmp-xETTca/ccache" inode=944
dev=fd:03 mode=0100600 ouid=0 ogid=0 rdev=00:00
obj=system_u:object_r:var_run_t:s0 nametype=NORMAL
node=ipa2.abaqis.com type=CWD msg=audit(1386103646.841:451293):  cwd="/"
node=ipa2.abaqis.com type=SYSCALL msg=audit(1386103646.841:451293):
arch=c000003e syscall=4 success=yes exit=0 a0=36e1fd5 a1=7fffcc37ea40
a2=7fffcc37ea40 a3=4 items=1 ppid=3731 pid=23883 auid=4294967295 uid=0
gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none)
ses=4294967295 comm="dogtag-ipa-retr" exe="/usr/bin/python"
subj=system_u:system_r:certmonger_t:s0 key=(null)
node=ipa2.abaqis.com type=AVC msg=audit(1386103646.841:451293): avc:
denied  { getattr } for  pid=23883 comm="dogtag-ipa-retr"
path="/var/run/certmonger/tmp-xETTca/ccache" dev=dm-3 ino=944
scontext=system_u:system_r:certmonger_t:s0
tcontext=system_u:object_r:var_run_t:s0 tclass=file

We get this after pumping through audit2allow:
#============= certmonger_t ==============
#!!!! The source type 'certmonger_t' can write to a 'file' of the
following types:
# certmonger_var_lib_t, certmonger_var_run_t, cert_t, dirsrv_config_t,
root_t, cluster_conf_t, cluster_var_lib_t, cluster_var_run_t

allow certmonger_t var_run_t:file { setattr read lock create write
getattr unlink open };

Running a restorecon on /var only gets one thing which doesn't seem
related:
restorecon reset /var/run/pki-ca.pid context
system_u:object_r:var_run_t:s0->system_u:object_r:pki_ca_var_run_t:s0

- -Erinn
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.15 (GNU/Linux)

iQEcBAEBAgAGBQJSoOSMAAoJENetaK3v/E7PB34H/2VSQ4m+gwnR2Zyu9/jWkTMz
Up9fwysNyPI9nFrlFfLavkixmZmOyU+GGMsqlBedfaKuc07JH8ZtYdVlKkZihcux
wCvjaGfxZdvXZ2RnzyldUO0vN+cEb2TxtLLAZzTfs6LyZiryLcs0R1SpKGpWjXdp
jqeZXyavXDBTEa3KrkKSTys24wXukHvaOkdrxcKOoryMLb3o+cTQfmORuuZlzJDO
pWX+QrZSMPTUBDaE8eCX8gFKbhHkRwerc3YB5jcbqZiRqBue2L1I9nJYZwWED+dV
GMXA1/wMCaB7PRRv8XvG8G7iCFBMp+88Td8GGCrgVxk4wnhjIzAtGxffbwf0Sd4=
=Lviw
-----END PGP SIGNATURE-----