[Freeipa-devel] [PATCHES] 241-253 CA certificate renewal

Thu Jun 26 11:36:26 UTC 2014

On 12.6.2014 09:49, Jan Cholasta wrote:
> On 20.5.2014 21:38, Rob Crittenden wrote:
>> Jan Cholasta wrote:
>>> On 25.4.2014 10:51, Jan Cholasta wrote:
>>>> On 24.4.2014 23:16, Rob Crittenden wrote:
>>>>> Jan Cholasta wrote:
>>>>>> On 10.4.2014 22:06, Rob Crittenden wrote:
>>>>>>> Some in-line, a whole ton of data appended to end.
>>>>>>>
>>>>>>> Jan Cholasta wrote:
>>>>>>>> On 7.4.2014 20:09, Rob Crittenden wrote:
>>>>>>>>> Rob Crittenden wrote:
>>>>>>>>>>
>>>>>>>>>> 247
>>>>>>>>>>
>>>>>>>>>> We've been burned by hardcoded timeouts in the past. Should
>>>>>>>>>> this be
>>>>>>>>>> configurable? This module doesn't currently do any logging but it
>>>>>>>>>> might
>>>>>>>>>> be worth spitting out a "waiting" message, at least for
>>>>>>>>>> debugging.
>>>>>>>>
>>>>>>>> Added a timeout argument.
>>>>>>>
>>>>>>> Did you forget to send this one, I didn't see an update to 247.
>>>>>>
>>>>>> Are you sure you have 247.1 (now 247.2)?
>>>>>>
>>>>>> I can see at
>>>>>> <http://www.redhat.com/archives/freeipa-devel/2014-April/msg00225.html>
>>>>>>
>>>>>> that I have sent the correct version of the patches.
>>>>>
>>>>> The call has a timeout, the callers don't use it. I guess it'll do for
>>>>> now, but these almost always come back to bite us.
>>>>
>>>> Well, I can add --certmonger-timeout option to ipa-cacert-manage, if
>>>> that's what you want.
>>>>
>>>>>
>>>>>>
>>>>>>>>>>
>>>>>>>>>> 251
>>>>>>>>>>
>>>>>>>>>> The tool should provide some feedback while it's running. For the
>>>>>>>>>> impatient (me) it takes a really long time and it's hard to know
>>>>>>>>>> what is
>>>>>>>>>> going on, something in between nothing and full debug output.
>>>>>>>>
>>>>>>>> Added some messages about what's going on.
>>>>>>>
>>>>>>> I dpn't see an update to 251 either.
>>>>>>
>>>>>> Please make sure you have 251.1 (now 251.2).
>>>>>
>>>>> There is a little bit more output but there are still very long
>>>>> periods
>>>>> of waiting between any visual activity, particularly when doing it
>>>>> on an
>>>>> IPA self-signed CA.
>>>>
>>>> This stuff takes time :-) What would you like to see in the output,
>>>> that's not already there?
>>>>
>>>>>>>
>>>>>>> I think the ipa-cacert-manage man page is missing one really
>>>>>>> important
>>>>>>> piece: why would you ever need to run this? And when?
>>>>>>
>>>>>> Added a paragraph about this.
>>>>>
>>>>> It's better, couple of comments:
>>>>>
>>>>> Add "the" in between renew and CA in "used to manually renew CA
>>>>> certificate of" and "When IPA CA...".
>>>>
>>>> OK.
>>>>
>>>>> I haven't had any luck renewing
>>>>> the CA certificate yet. I see that it is tracked now. I started moving
>>>>> the system clock forward in order to get to renewal and about the 3rd
>>>>> iteration the requests started failing with an XML error. Did you see
>>>>> this?
>>>>>
>>>>> [Thu Apr 21 11:08:49.929486 2016] [:error] [pid 11692] Traceback (most
>>>>> recent call last):
>>>>> [Thu Apr 21 11:08:49.929489 2016] [:error] [pid 11692]   File
>>>>> "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line
>>>>> 344, in
>>>>> wsgi_execute
>>>>> [Thu Apr 21 11:08:49.929493 2016] [:error] [pid 11692]     result =
>>>>> self.Command[name](*args, **options)
>>>>> [Thu Apr 21 11:08:49.929496 2016] [:error] [pid 11692]   File
>>>>> "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 436, in
>>>>> __call__
>>>>> [Thu Apr 21 11:08:49.929499 2016] [:error] [pid 11692]     ret =
>>>>> self.run(*args, **options)
>>>>> [Thu Apr 21 11:08:49.929503 2016] [:error] [pid 11692]   File
>>>>> "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 752, in
>>>>> run
>>>>> [Thu Apr 21 11:08:49.929506 2016] [:error] [pid 11692]     result =
>>>>> self.execute(*args, **options)
>>>>> [Thu Apr 21 11:08:49.929509 2016] [:error] [pid 11692]   File
>>>>> "/usr/lib/python2.7/site-packages/ipalib/plugins/cert.py", line
>>>>> 382, in
>>>>> execute
>>>>> [Thu Apr 21 11:08:49.929512 2016] [:error] [pid 11692]     result =
>>>>> api.Command['cert_show'](unicode(serial))['result']
>>>>> [Thu Apr 21 11:08:49.929516 2016] [:error] [pid 11692]   File
>>>>> "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 436, in
>>>>> __call__
>>>>> [Thu Apr 21 11:08:49.929519 2016] [:error] [pid 11692]     ret =
>>>>> self.run(*args, **options)
>>>>> [Thu Apr 21 11:08:49.930559 2016] [:error] [pid 11692]   File
>>>>> "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 752, in
>>>>> run
>>>>> [Thu Apr 21 11:08:49.930567 2016] [:error] [pid 11692]     result =
>>>>> self.execute(*args, **options)
>>>>> [Thu Apr 21 11:08:49.930570 2016] [:error] [pid 11692]   File
>>>>> "/usr/lib/python2.7/site-packages/ipalib/plugins/cert.py", line
>>>>> 514, in
>>>>> execute
>>>>> [Thu Apr 21 11:08:49.930573 2016] [:error] [pid 11692]
>>>>> result=self.Backend.ra.get_certificate(serial_number)
>>>>> [Thu Apr 21 11:08:49.930577 2016] [:error] [pid 11692]   File
>>>>> "/usr/lib/python2.7/site-packages/ipaserver/plugins/dogtag.py", line
>>>>> 1502, in get_certificate
>>>>> [Thu Apr 21 11:08:49.930580 2016] [:error] [pid 11692]
>>>>> parse_result
>>>>> = self.get_parse_result_xml(http_body, parse_display_cert_xml)
>>>>> [Thu Apr 21 11:08:49.930591 2016] [:error] [pid 11692]   File
>>>>> "/usr/lib/python2.7/site-packages/ipaserver/plugins/dogtag.py", line
>>>>> 1363, in get_parse_result_xml
>>>>> [Thu Apr 21 11:08:49.930594 2016] [:error] [pid 11692]     doc =
>>>>> etree.fromstring(xml_text, parser)
>>>>> [Thu Apr 21 11:08:49.930598 2016] [:error] [pid 11692]   File
>>>>> "lxml.etree.pyx", line 3032, in lxml.etree.fromstring
>>>>> (src/lxml/lxml.etree.c:68129)
>>>>> [Thu Apr 21 11:08:49.930601 2016] [:error] [pid 11692]   File
>>>>> "parser.pxi", line 1785, in lxml.etree._parseMemoryDocument
>>>>> (src/lxml/lxml.etree.c:102493)
>>>>> [Thu Apr 21 11:08:49.930604 2016] [:error] [pid 11692]   File
>>>>> "parser.pxi", line 1673, in lxml.etree._parseDoc
>>>>> (src/lxml/lxml.etree.c:101322)
>>>>> [Thu Apr 21 11:08:49.930607 2016] [:error] [pid 11692]   File
>>>>> "parser.pxi", line 1074, in lxml.etree._BaseParser._parseDoc
>>>>> (src/lxml/lxml.etree.c:96504)
>>>>> [Thu Apr 21 11:08:49.930611 2016] [:error] [pid 11692]   File
>>>>> "parser.pxi", line 582, in
>>>>> lxml.etree._ParserContext._handleParseResultDoc
>>>>> (src/lxml/lxml.etree.c:91308)
>>>>> [Thu Apr 21 11:08:49.930614 2016] [:error] [pid 11692]   File
>>>>> "parser.pxi", line 683, in lxml.etree._handleParseResult
>>>>> (src/lxml/lxml.etree.c:92494)
>>>>> [Thu Apr 21 11:08:49.930617 2016] [:error] [pid 11692]   File
>>>>> "parser.pxi", line 633, in lxml.etree._raiseParseError
>>>>> (src/lxml/lxml.etree.c:91957)
>>>>> [Thu Apr 21 11:08:49.930621 2016] [:error] [pid 11692] XMLSyntaxError:
>>>>> None
>>>>> [Thu Apr 21 11:08:49.930829 2016] [:error] [pid 11692] ipa: INFO:
>>>>> [xmlserver] host/lyra.greyoak.com at GREYOAK.COM:
>>>>> cert_request(u'MIIDmTCCAoECAQAwMTEUMBIGA1UECgwLR1JFWU9BSy5DT00xGTAXBgNVBAMMEGx5cmEuZ3JleW9hay5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDVw34/WP/pKg+kxmM7aDcjYsEFA9By4SyV1n4FeeDpr2gBulyjKNGTaxuoEssqYy33qVU7vxQ8WS9LtYt1eyAZrKLCVso1njtMZZ2mpymltRgRT+QFzMjwrcoqqGM9XWjVAMur/FJggVPxcpNXy2EUAiVcMEO4zCgxjkWjaXSs5jigvFO5wzolnQRYPECPhYuYwKpVRPwCsqpeiymfpiVuHt+oTHA9lNIGRWWxA+72NLFhrLBKaVaFDl4NCT6jJ71xZDXLQWQw1kAWvYKV22jCfDS/Yxdtyt3O0kUs8dHYClTgW4QN3bBQsgUaspHuWGdF6rwqmIihPUjq07fB6r8jAgMBAAGgggEhMCUGCSqGSIb3DQEJFDEYHhYAUwBlAHIAdgBlAHIALQBDAGUAcgB0MIH3BgkqhkiG9w0BCQ4xgekwgeYwDgYDVR0PAQEABAQDAgTwMIGBBgNVHREBAQAEdzB1oDEGCisGAQQBgjcUAgOgIwwhSFRUUC9seXJhLmdyZXlvYWsuY29tQEdSRVlPQUsuQ09NoEAGBisGAQUCAqA2MDSgDRsLR1JFWU9BSy5DT02hIzAhoAMCAQGhGjAYGwRIVFRQGxBseXJhLmdyZXlvYWsuY29tMCAGA1UdJQEBAAQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAMBgNVHRMBAf8EAjAAMCAGA1UdDgEBAAQWBBQ2k1wey/NlwORBA7I+O26IX0WyGTANBgkqhkiG9w0BAQsFAAOCAQEABUStRJyl5DBTpEOdKOXCnhSdRuElwv64XLIX47B1DVsiI9tL806nsLH16XSFIZqo1HWXxDU90/Wm!
 8!
>>>>>
> V!
>>   P!
>>>>>
>>> Z!
>>>>>
>>>> gm!
>>>>>
>>>>> 3VCtgMvPVk
>>>>> 3k4qYBz6/2B8PEeQY2/W5CULkfjqJhDxr0qodiYAc8GOyHMDpymfC3+QUIXkmoy94USRS2x8CMvzq8h1tpBPcXAei6waohTJtO33o79iVNbeLIif3RD22dghPx3JvEB4FXWQv6IylXGyJb6NRRneI4R8Ko0xCA9xiyPegfDgiQEUUSCtJ/Qr9/OpytFgrpJHSTd8n9DzLbRO5FQW4yS45A8xp5WkJCU5IslIon6luf9v5eNCVsIp7EPgaQ==',
>>>>>
>>>>>
>>>>>
>>>>> principal=u'HTTP/lyra.greyoak.com at GREYOAK.COM', add=True,
>>>>> version=u'2.51'): XMLSyntaxError
>>>>
>>>> I have never seen this. The error message does not say much... Is there
>>>> anything interesting in other logs?
>>>
>>> I was able to get the CA certificate to be renewed after moving system
>>> time forward step by step.
>>>
>>> One thing I haven't noticed before is that the renewed certificate's
>>> validity never exceeds that of the original certificate. This is most
>>> likely Dogtag issue (something along the lines of "certificate validity
>>> cannot exceed validity of the CA certificate", except it shouldn't apply
>>> to the CA certificate itself).
>>>
>>> There were other issues here and there, all of them were caused by race
>>> conditions between concurrent renewals (unreachable CA, XML syntax
>>> errors, etc. because Dogtag was stopped by stop_pkicad in another
>>> request, CMS internal error because it used old subsystem cert to
>>> authenticate to LDAP while the cert was being renewed, etc.) and all of
>>> them could be fixed by restarting relevant IPA services and resubmitting
>>> the requests manually. Some synchronization is really missing there.
>>
>> I hadn't noticed that, but my CA was issued externally so I expected
>> this. I also saw the bumps during renewal but things always tended to
>> smooth out, with the errors generally restricted to restarts and
>> certmonger. This backtrace was the only thing that really stood out.
>> IIRC at this point things were pretty much blocked.
>>
>> In any case, these patches basically seem to work. I never did work out
>> whether the above error was due to dogtag, IPA or something else.
>>
>> rob
>
> Rebased the patches on top of current master.
>
> Give up retrieving certificate from LDAP in patch 265 after a few
> unsuccessful attempts. This is to prevent certmonger requests from
> staying in CA_WORKING state forever when you manually resubmit a request.
>
> Added patch 266 which adds ACIs missing after the permission refactoring.

Rebased again.

Converted all permissions to managed permissions.

Added dependency on certmonger >= 0.74 in patch 251, because CSR export 
is broken with older versions. There is an update to certmonger 0.75.5 
for F20: 
<https://admin.fedoraproject.org/updates/FEDORA-2014-7529/certmonger-0.75.5-1.fc20>. 
(It segfaults for me during server install, I and Nalin are investigating.)

-- 
Jan Cholasta
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-241.4-Add-function-for-checking-if-certificate-is-self-sig.patch
Type: text/x-patch
Size: 895 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140626/5394223c/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-242.4-Support-CA-certificate-renewal-in-dogtag-ipa-ca-rene.patch
Type: text/x-patch
Size: 3220 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140626/5394223c/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-243.4-Allow-IPA-master-hosts-to-update-CA-certificate-in-L.patch
Type: text/x-patch
Size: 1077 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140626/5394223c/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-244.4-Automatically-update-CA-certificate-in-LDAP-on-renew.patch
Type: text/x-patch
Size: 2383 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140626/5394223c/attachment-0003.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-245.4-Track-CA-certificate-using-dogtag-ipa-ca-renew-agent.patch
Type: text/x-patch
Size: 5097 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140626/5394223c/attachment-0004.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-246.4-Add-method-for-setting-CA-renewal-master-in-LDAP-to-.patch
Type: text/x-patch
Size: 2471 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140626/5394223c/attachment-0005.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-247.4-Provide-additional-functions-to-ipapython.certmonger.patch
Type: text/x-patch
Size: 2097 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140626/5394223c/attachment-0006.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-248.4-Move-external-cert-validation-from-ipa-server-instal.patch
Type: text/x-patch
Size: 5954 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140626/5394223c/attachment-0007.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-249.4-Add-method-for-verifying-CA-certificates-to-NSSDatab.patch
Type: text/x-patch
Size: 2034 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140626/5394223c/attachment-0008.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-250.4-Add-permissions-for-CA-certificate-renewal.patch
Type: text/x-patch
Size: 4088 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140626/5394223c/attachment-0009.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-251.4-Add-CA-certificate-management-tool-ipa-cacert-manage.patch
Type: text/x-patch
Size: 17465 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140626/5394223c/attachment-0010.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-252.4-Alert-user-when-externally-signed-CA-is-about-to-exp.patch
Type: text/x-patch
Size: 1711 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140626/5394223c/attachment-0011.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-253.4-Load-sysupgrade.state-on-demand.patch
Type: text/x-patch
Size: 1341 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140626/5394223c/attachment-0012.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-262.3-Pick-new-CA-renewal-master-when-deleting-a-replica.patch
Type: text/x-patch
Size: 3778 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140626/5394223c/attachment-0013.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-263.2-Remove-master-ACIs-when-deleting-a-replica.patch
Type: text/x-patch
Size: 2614 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140626/5394223c/attachment-0014.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-264.2-Do-not-use-ldapi-in-certificate-renewal-scripts.patch
Type: text/x-patch
Size: 12106 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140626/5394223c/attachment-0015.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-265.2-Check-that-renewed-certificates-coming-from-LDAP-are.patch
Type: text/x-patch
Size: 2898 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140626/5394223c/attachment-0016.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-266.1-Allow-IPA-master-hosts-to-read-and-update-IPA-master.patch
Type: text/x-patch
Size: 3191 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20140626/5394223c/attachment-0017.bin>