[Freeipa-devel] [PATCH] Patch to allow IPA to work with dogtag 10 on f18
Rob Crittenden
rcritten at redhat.com
Tue Sep 11 18:45:20 UTC 2012
Petr Viktorin wrote:
> On 09/11/2012 04:38 PM, Rob Crittenden wrote:
>> Ade Lee wrote:
>>> On Tue, 2012-09-11 at 08:59 -0400, Rob Crittenden wrote:
>>>> Petr Viktorin wrote:
>>>>> On 09/11/2012 04:04 AM, Ade Lee wrote:
>>>>>> On Mon, 2012-09-10 at 16:58 -0400, Rob Crittenden wrote:
>>>>>>> Petr Viktorin wrote:
>>>>>>>> Attaching rebased and squashed patches. I've done some testing with
>>>>>>>> them
>>>>>>>> but please test some more.
>>>>>>>>
>>>>>>>
>>>>>>> Most of these aren't IPA issues, but dogtag issues. I'll try to
>>>>>>> split
>>>>>>> them out.
>>>>>>>
>>>>>>> IPA:
>>>>>>>
>>>>>>> For the configuration files in install/conf to be updated at rpm
>>>>>>> update
>>>>>>> time the VERSION needs to be incremented.
>>>>>
>>>>> These files should stay the same since on upgrade we're still using a
>>>>> Dogtag 9 style instance. The Dogtag 10 ports are only used in new
>>>>> installs.
>>>>>
>>>>>>> The ipa package lacks any updated dogtag dependencies, so I abused
>>>>>>> it.
>>>>>
>>>>> What should the updated dependencies be? Since it should work with
>>>>> both
>>>>> dogtag 9 and 10, I don't see how they should change.
>>>>
>>>> I don't know either, but we need to prevent people from installing
>>>> incompatible package combinations.
>>>>
>>> Would'nt the Conflicts: ipa < 3.0 in pki-ca mentioned below satisfy this
>>> requirement? The main concern is that you must have ipa 3.0 if you have
>>> dogtag 10.
>>>
>>> Given that dogtag is consumed by IPA though, it makes more sense to put
>>> the relevant conflicts in IPA rather than in dogtag. So in this case,
>>> that would mean putting Conflicts: pki-ca >= 10.0 in IPA 2.x.
>>> Recall that dogtag 10 will only be officially available in f18+.
>>
>> That isn't enough. If a F-17 user with IPA 2.2 installed upgrades to
>> F-18 they would be able to install dogtag 10 and blow up their IPA
>> server.
>>
>>>
>>>>>>> I installed IPA with dogtag 9 and created a replica.
>>>>>>>
>>>>>>> I updated the IPA bits, that worked fine.
>>>>>>>
>>>>>>> I updated to dogtag 10 and now the CA doesn't work on the master,
>>>>>>> including starting the dogtag instance. Note that the rpm update
>>>>>>> process
>>>>>>> worked, no notice that the CA service didn't restart.
>>>>>>>
>>>>>> Did you try to restart the CA with selinux in permissive mode?
>>>>>> This is
>>>>>> still required right now until I get the selinux policy all
>>>>>> straightened
>>>>>> out.
>>>>>>
>>>>>> There is also a separate dogtag ticket (which is currently being
>>>>>> worked
>>>>>> on) to restart existing dogtag instances when dogtag is upgraded from
>>>>>> 9->10.
>>>>>
>>>>> In permissive mode, this upgrade works for me.
>>>>
>>>> I was in enforcing mode but saw no AVCs. What is the ETA on fixing
>>>> this?
>>>>
>>>
>>> Within the next week or two, I need to finish the IPA merge database
>>> patch first, and then co-ordinate changes with the selinux guys.
>>>
>>>>>
>>>>>
>>>>> Sometimes I do get this error intermittently:
>>>>>
>>>>> ipa: ERROR: Certificate operation cannot be completed: Unable to
>>>>> communicate with CMS (Service Temporarily Unavailable)
>>>>>
>>>>> Usually, waiting a couple of minutes clears this up. Perhaps we are
>>>>> not
>>>>> doing startup detection correctly. Ade mentioned that waiting for
>>>>> ports
>>>>> may not be ideal. How do we know if Dogtag is initialized?
>>>>
>>>> Years ago we had discussed with Andrew and Matt creating a URI that can
>>>> be queried to determine dogtag status. I don't think that ever went
>>>> anywhere.
>>>>
>>> Petr, this happens only on reboot, right? And not on regular "service
>>> ipa restart"?
>
> I've now seen it happen right after a 9 → 10 upgrade.
>
>>> Yeah, I remember this conversation - and even created a bug for it at
>>> some point. This went away because the mechanism you were using seemed
>>> to be working. The timing may be very different now with tomcat 7 and
>>> systemd. I'll open a dogtag trac ticket for this.
>>
>> Ok.
>>
>>>
>>>>>
>>>>>>> Uninstalling failed because it tried to run pkidestroy and not
>>>>>>> pkiremove.
>>>>>
>>>>> I was under the impression that pkidestroy was the correct command to
>>>>> remove an upgraded instance. I'll check with Ade.
>>>>>
>>>>>> I'll test this too.
>>>>>>
>>>>>>> The contents of the file passed to pkispawn should be logged so we
>>>>>>> can
>>>>>>> see exactly what was passed in.
>>>>>>>
>>>>>> Its a pretty big file. You might want to only log the modifications.
>>>>>> Or save the file somewhere.
>>>>>
>>>>> Our logs are pretty verbose, so that shouldn't be a problem. I'll
>>>>> put it
>>>>> in the next version of the patch.
>>>>
>>>> The question to ask is: would you need the contents of this file if all
>>>> you got were logs and needed to evaluate why installation failed? In
>>>> most cases this is yes.
>>>>
>>> Up to you guys. There is a patch I am working on in which I will be
>>> logging the object that is being passed to the server from pkispawn.
>>> That - and the diffs to the standard config file as I mentioned above -
>>> will likely be sufficient to debug almost all cases.
>>>
>>> Make sure not to log any passwords.
>>>
>
> Thanks for the catch. Attaching updated patch that sanitizes the passwords.
>
>>>>>>> DOGTAG:
>>>>>>>
>>>>>>> When upgrading using the dogtag-devel repo I had to specify
>>>>>>> pki-tools.x86_64 otherwise it tried to install both 32 and 64-bit
>>>>>>> versions (and failed).
>>>>>>>
>>>>>>> I ended up running: yum update pki-ca tomcatjss pki-tools.x86_64
>>>>>>> --enablerepo=dogtag-devel --enablerepo=updates-testing
>>>>>>>
>>>>>> We'll look into this. I think I know why this happens.
>>>>>>
>>>>>>> What happens if someone manually upgrades pki-ca without first
>>>>>>> updating
>>>>>>> ipa? I think that pki-ca is going to need a Conflicts ipa < 3.0 in
>>>>>>> it.
>>>>>>
>>>>>> We can add that.
>>>>>>
>>>>>>> certificate renewal failed. I spent far too long trying to figure
>>>>>>> out
>>>>>>> why tomcat wasn't listening on port 9180 but failed. I think 9180 is
>>>>>>> actually the old server, right? So another missing dependency on a
>>>>>>> fixed
>>>>>>> certmonger?
>>>>>>>
>>>>>>> The best I could find was the certmonger error:
>>>>>>>
>>>>>>> ca-error: Error 7 connecting to
>>>>>>> http://edsel.example.com:9180/ca/ee/ca/profileSubmit: Couldn't
>>>>>>> connect
>>>>>>> to server.
>>>>>>>
>>>>>
>>>>> I'll set the certmonger min version to 0.60 as per Nalin's mail.
>>>>
>>>> Ok.
>>>>
>>>>>> Is this cert renewal on a dogtag 10 instance? Or the upgraded
>>>>>> dogtag 9?
>>>>>> If its dogtag 10, perhaps you do not have the certmonger version that
>>>>>> has the relevant fix? If its dogtag 9, then we need to figure out
>>>>>> whats
>>>>>> going on. That reminds me - I need to file a bug to allow
>>>>>> certmonger to
>>>>>> talk to the newly defined dogtag ports. Do you have selinux
>>>>>> permissive?
>>>>>>
>>>>>>> There is no man page for pkispawn/pkidestroy :-( According to the
>>>>>>> FHS
>>>>>>> these should not be in /bin but in /usr/sbin (not end-user
>>>>>>> commands).
>>>>>>>
>>>>>> There is a trac ticket open for man pages for pkispawn and
>>>>>> pkidestroy.
>>>>>> We plan to complete this ticket by the time f18 is released.
>>>>>>
>>>>>> We'll look into the location of pkispawn/pkicreate.
>>>>>>
>>>>>>> The output of pkicreate/pkisilent was really terrible and not
>>>>>>> usable at
>>>>>>> all so we didn't display it when failures occurred. It looks like
>>>>>>> that
>>>>>>> has been addressed, at least for the case where a CA is already
>>>>>>> configured and you try to install IPA. Perhaps we should capture
>>>>>>> stderr
>>>>>>> and display that instead of the command-line of pkispawn? Again, a
>>>>>>> man
>>>>>>> page would help with the integration.
>>>>>>>
>>>>>>> 2012-09-10T20:51:45Z DEBUG [2/18]: configuring certificate server
>>>>>>> instance
>>>>>>> 2012-09-10T20:51:45Z DEBUG args=/bin/pkispawn -s CA -f
>>>>>>> /tmp/tmp_Urraq
>>>>>>> 2012-09-10T20:51:45Z DEBUG stdout=
>>>>>>> 2012-09-10T20:51:45Z DEBUG stderr=pkispawn : ERROR ....... PKI
>>>>>>> subsystem 'CA' for instance 'pki-tomcat' already exists!
>>>>>>>
>>>>>>> 2012-09-10T20:51:45Z CRITICAL failed to configure ca instance
>>>>>>> Command
>>>>>>> '/bin/pkispawn -s CA -f /tmp/tmp_Urraq' returned non-zero exit
>>>>>>> status 1
>>>>>>>
>>>>>> That may be a good idea. Of course. thats an IPA thing, right?
>>>>
>>>> Right, not a show-stopper for this but a new ticket should be opened to
>>>> track it.
>
> https://fedorahosted.org/freeipa/ticket/3072
Thanks.
I tested a 2.2 -> 3.0 upgrade with dogtag 9 and it seems to have failed
to restart something:
[ post rpm -Uvh dist/rpms/freeipa*.rpm ]
[root at edsel freeipa]# ipa cert-show 1
ipa: ERROR: Certificate operation cannot be completed: Unable to
communicate with CMS (Service Temporarily Unavailable)
[root at edsel freeipa]# ipactl restart
Restarting Directory Service
Restarting KDC Service
Restarting KPASSWD Service
Restarting MEMCACHE Service
Restarting HTTP Service
Restarting CA Service
[root at edsel freeipa]# ipa cert-show 1
Certificate: MIIDjTCCAnWgAwIBAgIBATANBgk
...
The apache log had this:
[Tue Sep 11 14:35:42 2012] [error] (111)Connection refused: proxy: AJP:
attempt to connect to 127.0.0.1:9447 (localhost) failed
[Tue Sep 11 14:35:42 2012] [error] ap_proxy_connect_backend disabling
worker for (localhost)
[Tue Sep 11 14:35:42 2012] [error] proxy: AJP: failed to make connection
to backend: localhost
So I'm gathering that dogtag didn't restart properly, but I'm just
guessing. It could have been the PKI-IPA ds instance too, I'm not sure
where to check.
I also noticed this in /var/log/ipaupgrade.log:
2012-09-11T18:28:22Z DEBUG args=/bin/systemctl start certmonger.service
2012-09-11T18:28:22Z DEBUG stdout=
2012-09-11T18:28:22Z DEBUG stderr=
2012-09-11T18:28:22Z DEBUG args=/bin/systemctl is-active certmonger.service
2012-09-11T18:28:22Z DEBUG stdout=active
...
... [ snip certutil output ]
...
2012-09-11T18:28:52Z DEBUG args=/usr/bin/getcert start-tracking -d
/var/lib/pki-ca/alias -n auditSigningCert cert-pki-ca -c
dogtag-ipa-renew-agent -C /usr/lib64/ipa/certmonger/renew_ca_cert
"auditSigningCert cert-pki-ca" -P XXXXXXXX
2012-09-11T18:28:52Z DEBUG stdout=Please verify that the certmonger
service is still running.
2012-09-11T18:28:52Z DEBUG stderr=
2012-09-11T18:28:52Z INFO File
"/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py",
line 614, in run_script
return_value = main_function()
File "/usr/sbin/ipa-upgradeconfig", line 540, in main
enable_certificate_renewal(api.env.realm)
File "/usr/sbin/ipa-upgradeconfig", line 455, in
enable_certificate_renewal
ca.configure_renewal()
File
"/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line
1298, in configure_renewal
self.dogtag_constants.ALIAS_DIR, 'renew_ca_cert "%s"' % nickname)
File "/usr/lib/python2.7/site-packages/ipapython/certmonger.py", line
394, in dogtag_start_tracking
(stdout, stderr, returncode) = ipautil.run(args, nolog=[pin])
File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line
309, in run
raise CalledProcessError(p.returncode, args)
2012-09-11T18:28:52Z INFO The ipa-upgradeconfig command failed,
exception: CalledProcessError: Command '/usr/bin/getcert start-tracking
-d /var/lib/pki-ca/alias -n auditSigningCert cert-pki-ca -c
dogtag-ipa-renew-agent -C /usr/lib64/ipa/certmonger/renew_ca_cert
"auditSigningCert cert-pki-ca" -P XXXXXXXX' returned non-zero exit status 1
I'm not sure if this is related to this patch or not. If it isn't can
you file a new ticket on it, or add it the 2.2 -> 3.0 upgrade ticket?
rob
More information about the Freeipa-devel
mailing list