[Freeipa-devel] [PATCHES] 241-253 CA certificate renewal

Fri Jun 27 11:06:53 UTC 2014

On 26.6.2014 20:05, Rob Crittenden wrote:
> Jan Cholasta wrote:
>> On 12.6.2014 09:49, Jan Cholasta wrote:
>>> On 20.5.2014 21:38, Rob Crittenden wrote:
>>>> Jan Cholasta wrote:
>>>>> On 25.4.2014 10:51, Jan Cholasta wrote:
>>>>>> On 24.4.2014 23:16, Rob Crittenden wrote:
>>>>>>> Jan Cholasta wrote:
>>>>>>>> On 10.4.2014 22:06, Rob Crittenden wrote:
>>>>>>>>> Some in-line, a whole ton of data appended to end.
>>>>>>>>>
>>>>>>>>> Jan Cholasta wrote:
>>>>>>>>>> On 7.4.2014 20:09, Rob Crittenden wrote:
>>>>>>>>>>> Rob Crittenden wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> 247
>>>>>>>>>>>>
>>>>>>>>>>>> We've been burned by hardcoded timeouts in the past. Should
>>>>>>>>>>>> this be
>>>>>>>>>>>> configurable? This module doesn't currently do any logging
>>>>>>>>>>>> but it
>>>>>>>>>>>> might
>>>>>>>>>>>> be worth spitting out a "waiting" message, at least for
>>>>>>>>>>>> debugging.
>>>>>>>>>>
>>>>>>>>>> Added a timeout argument.
>>>>>>>>>
>>>>>>>>> Did you forget to send this one, I didn't see an update to 247.
>>>>>>>>
>>>>>>>> Are you sure you have 247.1 (now 247.2)?
>>>>>>>>
>>>>>>>> I can see at
>>>>>>>> <http://www.redhat.com/archives/freeipa-devel/2014-April/msg00225.html>
>>>>>>>>
>>>>>>>>
>>>>>>>> that I have sent the correct version of the patches.
>>>>>>>
>>>>>>> The call has a timeout, the callers don't use it. I guess it'll do
>>>>>>> for
>>>>>>> now, but these almost always come back to bite us.
>>>>>>
>>>>>> Well, I can add --certmonger-timeout option to ipa-cacert-manage, if
>>>>>> that's what you want.
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 251
>>>>>>>>>>>>
>>>>>>>>>>>> The tool should provide some feedback while it's running. For
>>>>>>>>>>>> the
>>>>>>>>>>>> impatient (me) it takes a really long time and it's hard to know
>>>>>>>>>>>> what is
>>>>>>>>>>>> going on, something in between nothing and full debug output.
>>>>>>>>>>
>>>>>>>>>> Added some messages about what's going on.
>>>>>>>>>
>>>>>>>>> I dpn't see an update to 251 either.
>>>>>>>>
>>>>>>>> Please make sure you have 251.1 (now 251.2).
>>>>>>>
>>>>>>> There is a little bit more output but there are still very long
>>>>>>> periods
>>>>>>> of waiting between any visual activity, particularly when doing it
>>>>>>> on an
>>>>>>> IPA self-signed CA.
>>>>>>
>>>>>> This stuff takes time :-) What would you like to see in the output,
>>>>>> that's not already there?
>>>>>>
>>>>>>>>>
>>>>>>>>> I think the ipa-cacert-manage man page is missing one really
>>>>>>>>> important
>>>>>>>>> piece: why would you ever need to run this? And when?
>>>>>>>>
>>>>>>>> Added a paragraph about this.
>>>>>>>
>>>>>>> It's better, couple of comments:
>>>>>>>
>>>>>>> Add "the" in between renew and CA in "used to manually renew CA
>>>>>>> certificate of" and "When IPA CA...".
>>>>>>
>>>>>> OK.
>>>>>>
>>>>>>> I haven't had any luck renewing
>>>>>>> the CA certificate yet. I see that it is tracked now. I started
>>>>>>> moving
>>>>>>> the system clock forward in order to get to renewal and about the 3rd
>>>>>>> iteration the requests started failing with an XML error. Did you see
>>>>>>> this?
>>>>>>>
>>>>>>> [Thu Apr 21 11:08:49.929486 2016] [:error] [pid 11692] Traceback
>>>>>>> (most
>>>>>>> recent call last):
>>>>>>> [Thu Apr 21 11:08:49.929489 2016] [:error] [pid 11692]   File
>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line
>>>>>>> 344, in
>>>>>>> wsgi_execute
>>>>>>> [Thu Apr 21 11:08:49.929493 2016] [:error] [pid 11692]     result =
>>>>>>> self.Command[name](*args, **options)
>>>>>>> [Thu Apr 21 11:08:49.929496 2016] [:error] [pid 11692]   File
>>>>>>> "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 436, in
>>>>>>> __call__
>>>>>>> [Thu Apr 21 11:08:49.929499 2016] [:error] [pid 11692]     ret =
>>>>>>> self.run(*args, **options)
>>>>>>> [Thu Apr 21 11:08:49.929503 2016] [:error] [pid 11692]   File
>>>>>>> "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 752, in
>>>>>>> run
>>>>>>> [Thu Apr 21 11:08:49.929506 2016] [:error] [pid 11692]     result =
>>>>>>> self.execute(*args, **options)
>>>>>>> [Thu Apr 21 11:08:49.929509 2016] [:error] [pid 11692]   File
>>>>>>> "/usr/lib/python2.7/site-packages/ipalib/plugins/cert.py", line
>>>>>>> 382, in
>>>>>>> execute
>>>>>>> [Thu Apr 21 11:08:49.929512 2016] [:error] [pid 11692]     result =
>>>>>>> api.Command['cert_show'](unicode(serial))['result']
>>>>>>> [Thu Apr 21 11:08:49.929516 2016] [:error] [pid 11692]   File
>>>>>>> "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 436, in
>>>>>>> __call__
>>>>>>> [Thu Apr 21 11:08:49.929519 2016] [:error] [pid 11692]     ret =
>>>>>>> self.run(*args, **options)
>>>>>>> [Thu Apr 21 11:08:49.930559 2016] [:error] [pid 11692]   File
>>>>>>> "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 752, in
>>>>>>> run
>>>>>>> [Thu Apr 21 11:08:49.930567 2016] [:error] [pid 11692]     result =
>>>>>>> self.execute(*args, **options)
>>>>>>> [Thu Apr 21 11:08:49.930570 2016] [:error] [pid 11692]   File
>>>>>>> "/usr/lib/python2.7/site-packages/ipalib/plugins/cert.py", line
>>>>>>> 514, in
>>>>>>> execute
>>>>>>> [Thu Apr 21 11:08:49.930573 2016] [:error] [pid 11692]
>>>>>>> result=self.Backend.ra.get_certificate(serial_number)
>>>>>>> [Thu Apr 21 11:08:49.930577 2016] [:error] [pid 11692]   File
>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/plugins/dogtag.py", line
>>>>>>> 1502, in get_certificate
>>>>>>> [Thu Apr 21 11:08:49.930580 2016] [:error] [pid 11692]
>>>>>>> parse_result
>>>>>>> = self.get_parse_result_xml(http_body, parse_display_cert_xml)
>>>>>>> [Thu Apr 21 11:08:49.930591 2016] [:error] [pid 11692]   File
>>>>>>> "/usr/lib/python2.7/site-packages/ipaserver/plugins/dogtag.py", line
>>>>>>> 1363, in get_parse_result_xml
>>>>>>> [Thu Apr 21 11:08:49.930594 2016] [:error] [pid 11692]     doc =
>>>>>>> etree.fromstring(xml_text, parser)
>>>>>>> [Thu Apr 21 11:08:49.930598 2016] [:error] [pid 11692]   File
>>>>>>> "lxml.etree.pyx", line 3032, in lxml.etree.fromstring
>>>>>>> (src/lxml/lxml.etree.c:68129)
>>>>>>> [Thu Apr 21 11:08:49.930601 2016] [:error] [pid 11692]   File
>>>>>>> "parser.pxi", line 1785, in lxml.etree._parseMemoryDocument
>>>>>>> (src/lxml/lxml.etree.c:102493)
>>>>>>> [Thu Apr 21 11:08:49.930604 2016] [:error] [pid 11692]   File
>>>>>>> "parser.pxi", line 1673, in lxml.etree._parseDoc
>>>>>>> (src/lxml/lxml.etree.c:101322)
>>>>>>> [Thu Apr 21 11:08:49.930607 2016] [:error] [pid 11692]   File
>>>>>>> "parser.pxi", line 1074, in lxml.etree._BaseParser._parseDoc
>>>>>>> (src/lxml/lxml.etree.c:96504)
>>>>>>> [Thu Apr 21 11:08:49.930611 2016] [:error] [pid 11692]   File
>>>>>>> "parser.pxi", line 582, in
>>>>>>> lxml.etree._ParserContext._handleParseResultDoc
>>>>>>> (src/lxml/lxml.etree.c:91308)
>>>>>>> [Thu Apr 21 11:08:49.930614 2016] [:error] [pid 11692]   File
>>>>>>> "parser.pxi", line 683, in lxml.etree._handleParseResult
>>>>>>> (src/lxml/lxml.etree.c:92494)
>>>>>>> [Thu Apr 21 11:08:49.930617 2016] [:error] [pid 11692]   File
>>>>>>> "parser.pxi", line 633, in lxml.etree._raiseParseError
>>>>>>> (src/lxml/lxml.etree.c:91957)
>>>>>>> [Thu Apr 21 11:08:49.930621 2016] [:error] [pid 11692]
>>>>>>> XMLSyntaxError:
>>>>>>> None
>>>>>>> [Thu Apr 21 11:08:49.930829 2016] [:error] [pid 11692] ipa: INFO:
>>>>>>> [xmlserver] host/lyra.greyoak.com at GREYOAK.COM:
>>>>>>> cert_request(u'MIIDmTCCAoECAQAwMTEUMBIGA1UECgwLR1JFWU9BSy5DT00xGTAXBgNVBAMMEGx5cmEuZ3JleW9hay5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDVw34/WP/pKg+kxmM7aDcjYsEFA9By4SyV1n4FeeDpr2gBulyjKNGTaxuoEssqYy33qVU7vxQ8WS9LtYt1eyAZrKLCVso1njtMZZ2mpymltRgRT+QFzMjwrcoqqGM9XWjVAMur/FJggVPxcpNXy2EUAiVcMEO4zCgxjkWjaXSs5jigvFO5wzolnQRYPECPhYuYwKpVRPwCsqpeiymfpiVuHt+oTHA9lNIGRWWxA+72NLFhrLBKaVaFDl4NCT6jJ71xZDXLQWQw1kAWvYKV22jCfDS/Yxdtyt3O0kUs8dHYClTgW4QN3bBQsgUaspHuWGdF6rwqmIihPUjq07fB6r8jAgMBAAGgggEhMCUGCSqGSIb3DQEJFDEYHhYAUwBlAHIAdgBlAHIALQBDAGUAcgB0MIH3BgkqhkiG9w0BCQ4xgekwgeYwDgYDVR0PAQEABAQDAgTwMIGBBgNVHREBAQAEdzB1oDEGCisGAQQBgjcUAgOgIwwhSFRUUC9seXJhLmdyZXlvYWsuY29tQEdSRVlPQUsuQ09NoEAGBisGAQUCAqA2MDSgDRsLR1JFWU9BSy5DT02hIzAhoAMCAQGhGjAYGwRIVFRQGxBseXJhLmdyZXlvYWsuY29tMCAGA1UdJQEBAAQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAMBgNVHRMBAf8EAjAAMCAGA1UdDgEBAAQWBBQ2k1wey/NlwORBA7I+O26IX0WyGTANBgkqhkiG9w0BAQsFAAOCAQEABUStRJyl5DBTpEOdKOXCnhSdRuElwv64XLIX47B1DVsiI9tL806nsLH16XSFIZqo1HWXxDU90/!
 W!
>   m!
>>>>>>>
>> 8!
>>>>>>>
>>> V!
>>>>    P!
>>>>>>>
>>>>> Z!
>>>>>>>
>>>>>> gm!
>>>>>>>
>>>>>>> 3VCtgMvPVk
>>>>>>> 3k4qYBz6/2B8PEeQY2/W5CULkfjqJhDxr0qodiYAc8GOyHMDpymfC3+QUIXkmoy94USRS2x8CMvzq8h1tpBPcXAei6waohTJtO33o79iVNbeLIif3RD22dghPx3JvEB4FXWQv6IylXGyJb6NRRneI4R8Ko0xCA9xiyPegfDgiQEUUSCtJ/Qr9/OpytFgrpJHSTd8n9DzLbRO5FQW4yS45A8xp5WkJCU5IslIon6luf9v5eNCVsIp7EPgaQ==',
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> principal=u'HTTP/lyra.greyoak.com at GREYOAK.COM', add=True,
>>>>>>> version=u'2.51'): XMLSyntaxError
>>>>>>
>>>>>> I have never seen this. The error message does not say much... Is
>>>>>> there
>>>>>> anything interesting in other logs?
>>>>>
>>>>> I was able to get the CA certificate to be renewed after moving system
>>>>> time forward step by step.
>>>>>
>>>>> One thing I haven't noticed before is that the renewed certificate's
>>>>> validity never exceeds that of the original certificate. This is most
>>>>> likely Dogtag issue (something along the lines of "certificate validity
>>>>> cannot exceed validity of the CA certificate", except it shouldn't
>>>>> apply
>>>>> to the CA certificate itself).
>>>>>
>>>>> There were other issues here and there, all of them were caused by race
>>>>> conditions between concurrent renewals (unreachable CA, XML syntax
>>>>> errors, etc. because Dogtag was stopped by stop_pkicad in another
>>>>> request, CMS internal error because it used old subsystem cert to
>>>>> authenticate to LDAP while the cert was being renewed, etc.) and all of
>>>>> them could be fixed by restarting relevant IPA services and
>>>>> resubmitting
>>>>> the requests manually. Some synchronization is really missing there.
>>>>
>>>> I hadn't noticed that, but my CA was issued externally so I expected
>>>> this. I also saw the bumps during renewal but things always tended to
>>>> smooth out, with the errors generally restricted to restarts and
>>>> certmonger. This backtrace was the only thing that really stood out.
>>>> IIRC at this point things were pretty much blocked.
>>>>
>>>> In any case, these patches basically seem to work. I never did work out
>>>> whether the above error was due to dogtag, IPA or something else.
>>>>
>>>> rob
>>>
>>> Rebased the patches on top of current master.
>>>
>>> Give up retrieving certificate from LDAP in patch 265 after a few
>>> unsuccessful attempts. This is to prevent certmonger requests from
>>> staying in CA_WORKING state forever when you manually resubmit a request.
>>>
>>> Added patch 266 which adds ACIs missing after the permission refactoring.
>>
>> Rebased again.
>>
>> Converted all permissions to managed permissions.
>>
>> Added dependency on certmonger >= 0.74 in patch 251, because CSR export
>> is broken with older versions. There is an update to certmonger 0.75.5
>> for F20:
>> <https://admin.fedoraproject.org/updates/FEDORA-2014-7529/certmonger-0.75.5-1.fc20>.
>> (It segfaults for me during server install, I and Nalin are investigating.)
>>

Fixed in 0.75.6: 
<https://admin.fedoraproject.org/updates/FEDORA-2014-7529/certmonger-0.75.6-1.fc20>

>
> I need a full set of patches, 241-299 from the same tree. Trying to
> piece together more than 50 patches from three long threads is proving
> impossible.

OK, I will attach all the required unpushed patches in each of the 
subsequent threads from now on.

(All of the patches I sent yesterday are from the same tree, so I don't 
really get why it is impossible to put them together.)

>
> Provided some feedback on 295-299 in that thread but I can't get much
> applied so I can do any functional testing.
>
> rob

Updated rebased patches attached.

-- 
Jan Cholasta
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-241.5-Add-function-for-checking-if-certificate-is-self-sig.patch
Type: text/x-patch
Size: 895 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140627/4257d5f7/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-242.5-Support-CA-certificate-renewal-in-dogtag-ipa-ca-rene.patch
Type: text/x-patch
Size: 3220 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140627/4257d5f7/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-243.5-Allow-IPA-master-hosts-to-update-CA-certificate-in-L.patch
Type: text/x-patch
Size: 1077 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140627/4257d5f7/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-244.5-Automatically-update-CA-certificate-in-LDAP-on-renew.patch
Type: text/x-patch
Size: 2383 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140627/4257d5f7/attachment-0003.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-245.5-Track-CA-certificate-using-dogtag-ipa-ca-renew-agent.patch
Type: text/x-patch
Size: 5097 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140627/4257d5f7/attachment-0004.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-246.5-Add-method-for-setting-CA-renewal-master-in-LDAP-to-.patch
Type: text/x-patch
Size: 2471 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140627/4257d5f7/attachment-0005.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-247.5-Provide-additional-functions-to-ipapython.certmonger.patch
Type: text/x-patch
Size: 2097 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140627/4257d5f7/attachment-0006.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-248.5-Move-external-cert-validation-from-ipa-server-instal.patch
Type: text/x-patch
Size: 5954 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140627/4257d5f7/attachment-0007.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-249.5-Add-method-for-verifying-CA-certificates-to-NSSDatab.patch
Type: text/x-patch
Size: 2034 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140627/4257d5f7/attachment-0008.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-250.5-Add-permissions-for-CA-certificate-renewal.patch
Type: text/x-patch
Size: 4088 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140627/4257d5f7/attachment-0009.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-251.5-Add-CA-certificate-management-tool-ipa-cacert-manage.patch
Type: text/x-patch
Size: 17469 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140627/4257d5f7/attachment-0010.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-252.5-Alert-user-when-externally-signed-CA-is-about-to-exp.patch
Type: text/x-patch
Size: 1711 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140627/4257d5f7/attachment-0011.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-253.5-Load-sysupgrade.state-on-demand.patch
Type: text/x-patch
Size: 1341 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140627/4257d5f7/attachment-0012.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-262.4-Pick-new-CA-renewal-master-when-deleting-a-replica.patch
Type: text/x-patch
Size: 3778 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140627/4257d5f7/attachment-0013.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-263.3-Remove-master-ACIs-when-deleting-a-replica.patch
Type: text/x-patch
Size: 2614 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140627/4257d5f7/attachment-0014.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-264.3-Do-not-use-ldapi-in-certificate-renewal-scripts.patch
Type: text/x-patch
Size: 12106 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140627/4257d5f7/attachment-0015.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-265.3-Check-that-renewed-certificates-coming-from-LDAP-are.patch
Type: text/x-patch
Size: 2898 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140627/4257d5f7/attachment-0016.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-266.2-Allow-IPA-master-hosts-to-read-and-update-IPA-master.patch
Type: text/x-patch
Size: 3191 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140627/4257d5f7/attachment-0017.bin>