[Freeipa-devel] [PATCHES] Re: Changes to use a single database for dogtag and IPA

Wed Nov 21 12:46:22 UTC 2012

On 11/21/2012 10:54 AM, Petr Viktorin wrote:
> On 11/21/2012 10:48 AM, Martin Kosek wrote:
>> On 11/21/2012 10:46 AM, Martin Kosek wrote:
>>> On 11/20/2012 02:59 PM, Petr Viktorin wrote:
>>>> On 11/19/2012 03:14 PM, Petr Viktorin wrote:
>>>>> On 11/19/2012 02:09 PM, Martin Kosek wrote:
>>>>>> On 11/16/2012 02:25 PM, Martin Kosek wrote:
>>>>>>> On 11/16/2012 11:23 AM, Martin Kosek wrote:
>>>>>>>> On 11/15/2012 07:17 PM, Petr Viktorin wrote:
>>>>>>>>> On 11/15/2012 05:09 PM, Martin Kosek wrote:
>>>>>>>>>> On 11/15/2012 03:19 PM, Petr Viktorin wrote:
>>>>>>>>>>> Recently, the specfile changed (dce53e4) and the patch for
>>>>>>>>>>> changed Dogtag
>>>>>>>>>>> defaults made it to master independently (91e477b). Attaching
>>>>>>>>>>> rebased patch.
>>>>>>>>>>>
>>>>>>>>>>> Note that to continue development on f17, you will need to use the
>>>>>>>>>>> dogtag-devel
>>>>>>>>>>> repo:
>>>>>>>>>>>      sudo yum-config-manager
>>>>>>>>>>> --add-repo=http://nkinder.fedorapeople.org/dogtag-devel/dogtag-devel-fedora.repo
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 11/13/2012 03:57 PM, Petr Viktorin wrote:
>>>>>>>>>>> [...]
>>>>>>>>>>>>
>>>>>>>>>>>> For convenience, I've also pushed the changes to a personal
>>>>>>>>>>>> repository.
>>>>>>>>>>>> To fetch to branch "pviktori-dogtag-10" you can do:
>>>>>>>>>>>>
>>>>>>>>>>>>         git fetch -f git://github.com/encukou/freeipa.git
>>>>>>>>>>>> dogtag-10:pviktori-dogtag-10
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I started reviewing the patches, and found the first thing that looks
>>>>>>>>>> suspicious. I had IPA with 2 databases, then upgraded it to
>>>>>>>>>> single-database
>>>>>>>>>> IPA, the upgrade was OK.
>>>>>>>>>>
>>>>>>>>>> But when I uninstalled the IPA, PKI-IPA dirsrv instance was not
>>>>>>>>>> removed
>>>>>>>>>> because
>>>>>>>>>> when I installed single-db IPA afterwards, I had 2 dirsrv
>>>>>>>>>> instances running.
>>>>>>>>>
>>>>>>>>> You're right. This is an uninstaller error already present in 2.2:
>>>>>>>>> https://fedorahosted.org/freeipa/ticket/3258
>>>>>>>>>
>>>>>>>>> I'll start looking into it tomorrow, if nothing more important
>>>>>>>>> shows up.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks for the pointer. But this is definitely not a show stopper,
>>>>>>>> running
>>>>>>>> additional DS instance seems more or less benign and as you pointed
>>>>>>>> out, it is
>>>>>>>> rather an old bug.
>>>>>>>>
>>>>>>>> There are bigger issues. Now I focused on ipa-replica-manage and
>>>>>>>> ipa-csreplica-manage tools. ipa-replica-manage gets confused with the
>>>>>>>> additional replication agreements in IPA dirsrv instance (although
>>>>>>>> targeted to
>>>>>>>> nsDS5ReplicaRoot: o=ipaca).
>>>>>>>>
>>>>>>>> First scenario: 3 IPA servers with CA in this topology:
>>>>>>>>
>>>>>>>> B - A - C
>>>>>>>>
>>>>>>>> On A:
>>>>>>>> # ipa-replica-manage list `hostname`
>>>>>>>> vm-055.idm.lab.bos.redhat.com: replica
>>>>>>>> vm-070.idm.lab.bos.redhat.com: replica
>>>>>>>> vm-055.idm.lab.bos.redhat.com: replica
>>>>>>>> vm-070.idm.lab.bos.redhat.com: replica
>>>>>>>>
>>>>>>>> it should not display agreements that are for IPA only, not IPA CA
>>>>>>>> ones.
>>>>>>>>
>>>>>>>> Now, when I try to connect B to C, ipa-replica-manage succeeded:
>>>>>>>> [B] # ipa-replica-manage connect C
>>>>>>>> Connected 'B' to 'C'
>>>>>>>>
>>>>>>>> This changed the topology to:
>>>>>>>>        A
>>>>>>>>      /   \
>>>>>>>> B   -  C
>>>>>>>>
>>>>>>>> But ipa-csreplica-manage connect did not succeed then:
>>>>>>>> [B] # ipa-csreplica-manage connect C
>>>>>>>> Directory Manager password:
>>>>>>>>
>>>>>>>> This replication agreement already exists.
>>>>>>>>
>>>>>>>> Del command also failed for me:
>>>>>>>> [A] ipa-replica-manage del [C]
>>>>>>>>
>>>>>>>> Still trying to investigate why. If I manage to get some workable
>>>>>>>> fix during my
>>>>>>>> investigations, I will attach it later.
>>>>>>>>
>>>>>>>> Martin
>>>>>>>
>>>>>>> The fix for that for easier than expected. Attached patch restored
>>>>>>> the previous
>>>>>>> functionality for ipa-(cs)replica-manage. I tried that with all basic
>>>>>>> commands
>>>>>>> - add, del, connect, disconnect and it worked fine so far.
>>>>>>>
>>>>>>> But this was a case with all D10 masters, I will need to try if that
>>>>>>> flies with
>>>>>>> D9->D10 replicas or upgraded D9 masters.
>>>>>>>
>>>>>>> Martin
>>>>>>>
>>>>>>
>>>>>> I managed to create a 2.2 (F17) -> 3.1 (F18) replica, everything seem
>>>>>> to work
>>>>>> well. I just think we will need to also backport the previous patch at
>>>>>> least to
>>>>>> 3.0 and 2.2 versions to fix errors with ipa-replica-manage replication
>>>>>> management tool. I created a ticket for this purpose:
>>>>>>
>>>>>> https://fedorahosted.org/freeipa/ticket/3262
>>>>>>
>>>>>> Attaching a patch for IPA 2.2 branch in case somebody is also testing
>>>>>> it. With
>>>>>> this patch, I was able to list, force-sync, re-initialize, connect,
>>>>>> disconnect
>>>>>> from 2.2 to 3.1 replica without any errors.
>>>>>>
>>>>>> Martin
>>>>>>
>>>>>
>>>>> I checked ipa-csreplica-install, and found some more problems.
>>>>>
>>>>> In the "connect" and "disconnect" subcommands we now assume both masters
>>>>> use port 389 for the PKI DS. While the number is relatively easily
>>>>> parametrized, the assumption that both sides use the same port seems to
>>>>> run pretty deep.
>>>>> I'm working on a patch to detect if the remote server has merged DBs and
>>>>> use the appropriate port.
>>>>
>>>> Attaching patches to do this.
>>>> Patch 0100 adds an argument to IPAdmin to overrides guessing of the
>>>> protocol to
>>>> use.
>>>> Patch 0101 modifies ipa-csreplica-manage to figure out the ports of the DSs
>>>> involved and use those.
>>>>
>>>> Note that this touches code shared with ipa-replica-manage, please re-test
>>>> that
>>>> as well.
>>>>
>>>
>>> Works fine, I was able to create a network of few IPA 2.2 replicas and IPA 3.1
>>> replicas and use ipa-replica-manage and ipa-csreplica-manage to play with the
>>> agreements. No regression discovered so far.
>>>
>>> I just see that in patch 101 you touch setup_replication and force TLS as a
>>> default. But in this case, r_sslport parameter is never used and we can
>>> remove it.
>>>
>>> In 101, you also set LDAP+TLS as default connection protocol with
>>> +        super(CSReplicationManager, self).__init__(
>>> +            realm, hostname, dirman_passwd, port, starttls=True)
>>>                                                     ^^^^^^^^^^^^^
>>>
>>> Wouldn't we want to make LDAP+TLS as a default also in a bunch of
>>> ReplicationManager initializations in ipa-replica-manage? Otherwise, we use
>>> ldaps/SSL by default. AFAIU, LDAP+TLS is preferred over ldaps/SSL so this would
>>> be a good step to do. Adding Rob and Simo to CC to correct me if I miss
>>> anything and we want to keep using ldaps/SSL by default.
>> ... adding to CC now!
>>
> 
> Yes, that would be good. I think it's out of scope for this patchset, though.
> Do we have a bug for this already?
> I think John or I can include it in the
> https://fedorahosted.org/freeipa/ticket/2660 or
> https://fedorahosted.org/freeipa/ticket/2652 efforts.
> 

Right, I think we do not have to do this now. I would not include it in 2652,
it is too general. I would prefer a separate ticket (if we agree with this
change) or inclusion in 2660.

On a different note, I tried a scenario of 3.0 split instance IPA master and
3.1 single instance replica and I got an error again:

# ipa-replica-install replica-info-vm-055.idm.lab.bos.redhat.com.gpg --setup-ca
Directory Manager (existing master) password:
...
  [28/30]: enabling compatibility plugin
  [29/30]: tuning directory server
  [30/30]: configuring directory to start on boot
Done configuring directory server (dirsrv).
Configuring certificate server (pki-tomcatd): Estimated time 3 minutes 30 seconds
  [1/16]: creating certificate server user
  [2/16]: configuring certificate server instance

Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.

Unexpected error - see /var/log/ipareplica-install.log for details:
IOError: [Errno 2] No such file or directory:
'/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12'

# ipa-ca-install replica-info-vm-055.idm.lab.bos.redhat.com.gpg
...
Configuring certificate server (pki-tomcatd): Estimated time 3 minutes 30 seconds
  [1/15]: creating certificate server user
  [2/15]: configuring certificate server instance

Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.

Unexpected error - see /var/log/ipareplica-ca-install.log for details:
IOError: [Errno 2] No such file or directory:
'/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12'

# rpm -q pki-ca
pki-ca-10.0.0-0.52.b3.fc18.noarch

# tail -1 /var/log/pki/pki-tomcat/ca/system
3357.http-bio-8443-exec-1 - [21/Nov/2012:07:16:02 EST] [8] [3] In Ldap (bound)
connection pool to host vm-104.idm.lab.bos.redhat.com port 7389, Cannot connect
to LDAP server. Error: netscape.ldap.LDAPException: failed to connect to server
ldap://vm-104.idm.lab.bos.redhat.com:7389 (91)

But when I try an ldapsearch to this address, it works:
# ldapsearch -H "ldap://vm-104.idm.lab.bos.redhat.com:7389" -s base -b
"o=ipaca" -D "cn=Directory Manager" -x -w kokos123
# extended LDIF

# ipaca
dn: o=ipaca
objectClass: top
objectClass: organization
o: ipaca

Attaching relevant logs and CC-ing Ade, I think we will need his help on this one.

IMO it is important to have this scenario working too, F17 IPA users who'd
upgrade to F18 and try to setup a replica may hit the same issue.

Martin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: logs.tgz
Type: application/x-compressed-tar
Size: 39184 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20121121/ad2f90f4/attachment.bin>