[Freeipa-devel] certificate renewal

[Rob Crittenden]

Vaede, Roger (Contractor) wrote:
> I cannot upgrade to IPA 3.0 at this time, these are live machines.
> I only want to renew only the primary server the one that has an expired certificate.
> How can I tell if the server is running on CA?

If you do a search like this:

# ldapsearch -LLL -Y GSSAPI -b cn=masters,cn=ipa,cn=etc,dc=example,dc=com cn

You'll get a listing, per-master, of the services on it. Look for the CA 
service.

Some of the certificates are common among multiple CA replicas, and some 
are machine-specific. They can be renewed only one one master and then 
copied to the others, this is why I asked.

As you'll see, this process is rather hairy and prone to error. I've 
done this any number of times on an IPA 3.x server while I worked on 
automating it but AFAIK never on a 2.x one. In theory it should work the 
same. In theory.

The first thing you'll need to do is get a newer version of certmonger, 
at least certmonger-0.61-3.el6.

IIRC you are currently tracking a bunch of bogus certs in certmonger. 
I'd one by one stop tracking them:

# getcert list | grep Request

for each cert do

# getcert stop-tracking -i <request_id>

Next, you need to determine the PIN for the CA NSS database:

# grep internal= /var/lib/pki-ca/conf/password.conf

Now we need to tell certmonger about all the CA certificates it needs to 
renew

for nickname in "auditSigningCert cert-pki-ca" "ocspSigningCert 
cert-pki-ca" "subsystemCert cert-pki-ca" "Server-Cert cert-pki-ca"
do
     /usr/bin/getcert start-tracking -d /var/lib/pki-ca/alias -n 
\"${nickname}\" -c dogtag-ipa-renew-agent -P <internal pin>
done

We also need to renew the agent certificate that IPA uses to authenticate:

# /usr/bin/getcert start-tracking -d /etc/httpd/alias -n ipaCert -c 
dogtag-ipa-renew-agent -p /etc/httpd/alias/pwdfile.txt

Use "getcert list" to confirm that these 5 certs are now being tracked 
and note the Request IDs.

Now time for the wayback machine, go back to when these certs are all 
still valid. I'd recommend a day or two before the earliest expiration.

You can force a renewal with:

  # getcert resubmit -i <request_id>

Do this for each one. Each should end up in a MONITORING state.

Double-check that the audit certificate has a trust of u,u,Pu. If not 
you can fix this with:

# certutil -M -n "auditSigningCert cert-pki-ca" -d /var/lib/pki-ca/alias 
-t u,u,Pu

Ok, now things get really fun. The dogtag configuration file has a 
base64-encoded copy of most of these certificates in it. You'll need to 
update those by hand.

What I'd recommend is for each nickname in /var/lib/pki-ca/alias do:

certutil -L -d /var/lib/pki-ca/alias -n <nickname> -a > /tmp/nickname

Then edit /etc/pki-ca/CS.cfg and find the cert entry for each one and 
replace the blobs. The option names are like ca.audit_signing.cert, 
ca.ocsp_signing.cert, etc. It should be fairly straightforward. There 
are 5 certs you need to replace.

Here are the nicknames and values:

'auditSigningCert cert-pki-ca': 'ca.audit_signing.cert'
'ocspSigningCert cert-pki-ca': 'ca.ocsp_signing.cert'
'caSigningCert cert-pki-ca': 'ca.signing.cert'
'subsystemCert cert-pki-ca': 'ca.subsystem.cert'
'Server-Cert cert-pki-ca': 'ca.sslserver.cert'

Note that the output from certutil will be in PEM format, with headers 
and broken on a 64-character line. You'll need to drop the header/footer 
and make them into a single line.

Backing up this file in advance would be a good idea.

Now you can try to restart the CA to see what happens. It should come up 
fine:

# /sbin/service pki-cad restart

For the ipaCert value in /etc/httpd/alias you have another job to do. 
The certificates allowed to authenticate are stored in the CA LDAP 
database. You'll need to use ldapmodify to fix things up.

Start by looking at the new value for ipaCert. You need to do two things:

# certutil -L -d /etc/httpd/alias -n ipaCert

Note the value of serial number.

Next you need the base64-encoded value of the cert like before:

# certutil -L -d /etc/httpd/alias -n ipaCert -a

Again you'll need to drop the header/footer and combine this into a 
single line.

Next see what is already there with:

# ldapsearch -x -h localhost -p 7389 -D 'cn=directory manager' -W -b 
uid=ipara,ou=People,o=ipaca

You need to replace the serial number in the description attribute with 
the new one. The serial number is the 2nd number. The format of the 
description line is:

2;<serial number>;<issuer subject>;<subject>

The change is going to look something like:

# ldapmodify -x -h localhost -p 7389 -D 'cn=directory manager' -W
dn: uid=ipara,ou=People,o=ipaca
changetype: modify
add: usercertificate;binary
usercertificate;binary: MII...=
replace: description
description: 2;9;CN=Certificate Authority,O=EXAMPLE.COM;CN=IPA 
RA,O=EXAMPLE.COM
<extra blank line to finish>
^D to exit

Now tackle the Apache and 389-ds certs:

# /usr/bin/ipa-getcert start-tracking -d /etc/dirsrv/slapd-REALM -n 
Server-Cert -p /etc/dirsrv/slapd-REALM/pwdfile.txt

# /usr/bin/ipa-getcert start-tracking -d /etc/dirsrv/slapd-PKI-IPA -n 
Server-Cert -p /etc/dirsrv/slapd-PKI-IPA/pwdfile.txt

# /usr/bin/ipa-getcert start-tracking -d /etc/httpd/alias -n Server-Cert 
-p /etc/httpd/alias/pwdfile.txt

Use getcert to check the status of these and resubmit as needed. It may 
be that certmonger will notice they are near expiration time and renew 
automagically.

Restart those services:

# /sbin/service httpd restart
# /sbin/service dirsrv restart

Return to present day.

Now test to see if the new certs are working:

# kinit admin
# ipa cert-show 1

This simple command actually exercises everything fairly well.

If you've done everything right, and I managed to not miss a step, it 
should return the cert and you're done. Further testing might include 
adding a test host and generating a certificate for it.

In the future, when these certificates expire again, you'll still have 
to do some of this manual labor. The certificates themselves should 
renew ok, but you'll need to update the configuration files and the 
o=ipara entry manually.

rob

>
> Thanks for your help on this Rob.
>
> Regards
> Roger
>
> -----Original Message-----
> From: Rob Crittenden [mailto:rcritten at redhat.com]
> Sent: Wednesday, October 30, 2013 5:07 PM
> To: Vaede, Roger (Contractor); 'freeipa-devel at redhat.com'
> Subject: Re: [Freeipa-devel] certificate renewal
>
> Vaede, Roger (Contractor) wrote:
>> There are two location of the alias:
>> In the backup:  /etc/httpd/alias/
>> In the one that has expired:  /var/lib/pki-ca/alias
>
> These are separate entities and will need to be dealt with seprately.
> /var/lib/pki-ca/alias is the NSS database that contains the certificates for the CA itself. With the subsystem certs being expired it means that the CA itself is basically dead in the water because nothing can talk to it.
>
> We're really moving into some uncharted waters here. I don't really want to recommend upgrading to 3.0 just to get the certs renewed, though there are lots of other good reasons to upgrade. I don't know what will happen if we try to renew the certs using the 3.0 method on a 2.x server.
>
> So you've got two IPA masters. Are you running the CA on both or only on one? It makes a difference in how we do the renewal because it will be more complex if you have it on both (not a show stopper).
>
> I *think* we can use the latest certmonger to do the majority of the heavy lifting. The basic process will be to update those bits, go back in time, the tell it to start tracking the CA certs one by one and get them renewed. We'll need to do the same with some other certs, some of which are in /etc/httpd/alias and some of which are in /etc/dirsrv/slapd-REALM/. If you have another instance of the CA we'll need to extract some of the renewed certs and import them on the other side. Then we return to present time.
>
> So let me know what your environment looks like and I'll try to come up with some steps to do the renewal.
>
> rob
>