[Freeipa-devel] [PATCHES] Re: Changes to use a single database for dogtag and IPA

Thu Nov 22 16:54:49 UTC 2012

On 11/21/2012 01:46 PM, Martin Kosek wrote:
> On 11/21/2012 10:54 AM, Petr Viktorin wrote:
>> On 11/21/2012 10:48 AM, Martin Kosek wrote:
>>> On 11/21/2012 10:46 AM, Martin Kosek wrote:
>>>> On 11/20/2012 02:59 PM, Petr Viktorin wrote:
>>>>> On 11/19/2012 03:14 PM, Petr Viktorin wrote:
>>>>>> On 11/19/2012 02:09 PM, Martin Kosek wrote:
>>>>>>> On 11/16/2012 02:25 PM, Martin Kosek wrote:
>>>>>>>> On 11/16/2012 11:23 AM, Martin Kosek wrote:
>>>>>>>>> On 11/15/2012 07:17 PM, Petr Viktorin wrote:
>>>>>>>>>> On 11/15/2012 05:09 PM, Martin Kosek wrote:
>>>>>>>>>>> On 11/15/2012 03:19 PM, Petr Viktorin wrote:
>>>>>>>>>>>> Recently, the specfile changed (dce53e4) and the patch for
>>>>>>>>>>>> changed Dogtag
>>>>>>>>>>>> defaults made it to master independently (91e477b). Attaching
>>>>>>>>>>>> rebased patch.
>>>>>>>>>>>>
>>>>>>>>>>>> Note that to continue development on f17, you will need to use the
>>>>>>>>>>>> dogtag-devel
>>>>>>>>>>>> repo:
>>>>>>>>>>>>       sudo yum-config-manager
>>>>>>>>>>>> --add-repo=http://nkinder.fedorapeople.org/dogtag-devel/dogtag-devel-fedora.repo
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 11/13/2012 03:57 PM, Petr Viktorin wrote:
>>>>>>>>>>>> [...]
>>>>>>>>>>>>>
>>>>>>>>>>>>> For convenience, I've also pushed the changes to a personal
>>>>>>>>>>>>> repository.
>>>>>>>>>>>>> To fetch to branch "pviktori-dogtag-10" you can do:
>>>>>>>>>>>>>
>>>>>>>>>>>>>          git fetch -f git://github.com/encukou/freeipa.git
>>>>>>>>>>>>> dogtag-10:pviktori-dogtag-10
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I started reviewing the patches, and found the first thing that looks
>>>>>>>>>>> suspicious. I had IPA with 2 databases, then upgraded it to
>>>>>>>>>>> single-database
>>>>>>>>>>> IPA, the upgrade was OK.
>>>>>>>>>>>
>>>>>>>>>>> But when I uninstalled the IPA, PKI-IPA dirsrv instance was not
>>>>>>>>>>> removed
>>>>>>>>>>> because
>>>>>>>>>>> when I installed single-db IPA afterwards, I had 2 dirsrv
>>>>>>>>>>> instances running.
>>>>>>>>>>
>>>>>>>>>> You're right. This is an uninstaller error already present in 2.2:
>>>>>>>>>> https://fedorahosted.org/freeipa/ticket/3258
>>>>>>>>>>
>>>>>>>>>> I'll start looking into it tomorrow, if nothing more important
>>>>>>>>>> shows up.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks for the pointer. But this is definitely not a show stopper,
>>>>>>>>> running
>>>>>>>>> additional DS instance seems more or less benign and as you pointed
>>>>>>>>> out, it is
>>>>>>>>> rather an old bug.
>>>>>>>>>
>>>>>>>>> There are bigger issues. Now I focused on ipa-replica-manage and
>>>>>>>>> ipa-csreplica-manage tools. ipa-replica-manage gets confused with the
>>>>>>>>> additional replication agreements in IPA dirsrv instance (although
>>>>>>>>> targeted to
>>>>>>>>> nsDS5ReplicaRoot: o=ipaca).
>>>>>>>>>
>>>>>>>>> First scenario: 3 IPA servers with CA in this topology:
>>>>>>>>>
>>>>>>>>> B - A - C
>>>>>>>>>
>>>>>>>>> On A:
>>>>>>>>> # ipa-replica-manage list `hostname`
>>>>>>>>> vm-055.idm.lab.bos.redhat.com: replica
>>>>>>>>> vm-070.idm.lab.bos.redhat.com: replica
>>>>>>>>> vm-055.idm.lab.bos.redhat.com: replica
>>>>>>>>> vm-070.idm.lab.bos.redhat.com: replica
>>>>>>>>>
>>>>>>>>> it should not display agreements that are for IPA only, not IPA CA
>>>>>>>>> ones.
>>>>>>>>>
>>>>>>>>> Now, when I try to connect B to C, ipa-replica-manage succeeded:
>>>>>>>>> [B] # ipa-replica-manage connect C
>>>>>>>>> Connected 'B' to 'C'
>>>>>>>>>
>>>>>>>>> This changed the topology to:
>>>>>>>>>         A
>>>>>>>>>       /   \
>>>>>>>>> B   -  C
>>>>>>>>>
>>>>>>>>> But ipa-csreplica-manage connect did not succeed then:
>>>>>>>>> [B] # ipa-csreplica-manage connect C
>>>>>>>>> Directory Manager password:
>>>>>>>>>
>>>>>>>>> This replication agreement already exists.
>>>>>>>>>
>>>>>>>>> Del command also failed for me:
>>>>>>>>> [A] ipa-replica-manage del [C]
>>>>>>>>>
>>>>>>>>> Still trying to investigate why. If I manage to get some workable
>>>>>>>>> fix during my
>>>>>>>>> investigations, I will attach it later.
>>>>>>>>>
>>>>>>>>> Martin
>>>>>>>>
>>>>>>>> The fix for that for easier than expected. Attached patch restored
>>>>>>>> the previous
>>>>>>>> functionality for ipa-(cs)replica-manage. I tried that with all basic
>>>>>>>> commands
>>>>>>>> - add, del, connect, disconnect and it worked fine so far.
>>>>>>>>
>>>>>>>> But this was a case with all D10 masters, I will need to try if that
>>>>>>>> flies with
>>>>>>>> D9->D10 replicas or upgraded D9 masters.
>>>>>>>>
>>>>>>>> Martin
>>>>>>>>
>>>>>>>
>>>>>>> I managed to create a 2.2 (F17) -> 3.1 (F18) replica, everything seem
>>>>>>> to work
>>>>>>> well. I just think we will need to also backport the previous patch at
>>>>>>> least to
>>>>>>> 3.0 and 2.2 versions to fix errors with ipa-replica-manage replication
>>>>>>> management tool. I created a ticket for this purpose:
>>>>>>>
>>>>>>> https://fedorahosted.org/freeipa/ticket/3262
>>>>>>>
>>>>>>> Attaching a patch for IPA 2.2 branch in case somebody is also testing
>>>>>>> it. With
>>>>>>> this patch, I was able to list, force-sync, re-initialize, connect,
>>>>>>> disconnect
>>>>>>> from 2.2 to 3.1 replica without any errors.
>>>>>>>
>>>>>>> Martin
>>>>>>>
>>>>>>
>>>>>> I checked ipa-csreplica-install, and found some more problems.
>>>>>>
>>>>>> In the "connect" and "disconnect" subcommands we now assume both masters
>>>>>> use port 389 for the PKI DS. While the number is relatively easily
>>>>>> parametrized, the assumption that both sides use the same port seems to
>>>>>> run pretty deep.
>>>>>> I'm working on a patch to detect if the remote server has merged DBs and
>>>>>> use the appropriate port.
>>>>>
>>>>> Attaching patches to do this.
>>>>> Patch 0100 adds an argument to IPAdmin to overrides guessing of the
>>>>> protocol to
>>>>> use.
>>>>> Patch 0101 modifies ipa-csreplica-manage to figure out the ports of the DSs
>>>>> involved and use those.
>>>>>
>>>>> Note that this touches code shared with ipa-replica-manage, please re-test
>>>>> that
>>>>> as well.
>>>>>
>>>>
>>>> Works fine, I was able to create a network of few IPA 2.2 replicas and IPA 3.1
>>>> replicas and use ipa-replica-manage and ipa-csreplica-manage to play with the
>>>> agreements. No regression discovered so far.
>>>>
>>>> I just see that in patch 101 you touch setup_replication and force TLS as a
>>>> default. But in this case, r_sslport parameter is never used and we can
>>>> remove it.
>>>>
>>>> In 101, you also set LDAP+TLS as default connection protocol with
>>>> +        super(CSReplicationManager, self).__init__(
>>>> +            realm, hostname, dirman_passwd, port, starttls=True)
>>>>                                                      ^^^^^^^^^^^^^
>>>>
>>>> Wouldn't we want to make LDAP+TLS as a default also in a bunch of
>>>> ReplicationManager initializations in ipa-replica-manage? Otherwise, we use
>>>> ldaps/SSL by default. AFAIU, LDAP+TLS is preferred over ldaps/SSL so this would
>>>> be a good step to do. Adding Rob and Simo to CC to correct me if I miss
>>>> anything and we want to keep using ldaps/SSL by default.
>>> ... adding to CC now!
>>>
>>
>> Yes, that would be good. I think it's out of scope for this patchset, though.
>> Do we have a bug for this already?
>> I think John or I can include it in the
>> https://fedorahosted.org/freeipa/ticket/2660 or
>> https://fedorahosted.org/freeipa/ticket/2652 efforts.
>>
>
> Right, I think we do not have to do this now. I would not include it in 2652,
> it is too general. I would prefer a separate ticket (if we agree with this
> change) or inclusion in 2660.
>
> On a different note, I tried a scenario of 3.0 split instance IPA master and
> 3.1 single instance replica and I got an error again:
>
> # ipa-replica-install replica-info-vm-055.idm.lab.bos.redhat.com.gpg --setup-ca
> Directory Manager (existing master) password:
> ...
>    [28/30]: enabling compatibility plugin
>    [29/30]: tuning directory server
>    [30/30]: configuring directory to start on boot
> Done configuring directory server (dirsrv).
> Configuring certificate server (pki-tomcatd): Estimated time 3 minutes 30 seconds
>    [1/16]: creating certificate server user
>    [2/16]: configuring certificate server instance
>
> Your system may be partly configured.
> Run /usr/sbin/ipa-server-install --uninstall to clean up.
>
> Unexpected error - see /var/log/ipareplica-install.log for details:
> IOError: [Errno 2] No such file or directory:
> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12'
>
> # ipa-ca-install replica-info-vm-055.idm.lab.bos.redhat.com.gpg
> ...
> Configuring certificate server (pki-tomcatd): Estimated time 3 minutes 30 seconds
>    [1/15]: creating certificate server user
>    [2/15]: configuring certificate server instance
>
> Your system may be partly configured.
> Run /usr/sbin/ipa-server-install --uninstall to clean up.
>
> Unexpected error - see /var/log/ipareplica-ca-install.log for details:
> IOError: [Errno 2] No such file or directory:
> '/var/lib/pki/pki-tomcat/alias/ca_backup_keys.p12'
>
> # rpm -q pki-ca
> pki-ca-10.0.0-0.52.b3.fc18.noarch
>
> # tail -1 /var/log/pki/pki-tomcat/ca/system
> 3357.http-bio-8443-exec-1 - [21/Nov/2012:07:16:02 EST] [8] [3] In Ldap (bound)
> connection pool to host vm-104.idm.lab.bos.redhat.com port 7389, Cannot connect
> to LDAP server. Error: netscape.ldap.LDAPException: failed to connect to server
> ldap://vm-104.idm.lab.bos.redhat.com:7389 (91)
>
> But when I try an ldapsearch to this address, it works:
> # ldapsearch -H "ldap://vm-104.idm.lab.bos.redhat.com:7389" -s base -b
> "o=ipaca" -D "cn=Directory Manager" -x -w kokos123
> # extended LDIF
>
> # ipaca
> dn: o=ipaca
> objectClass: top
> objectClass: organization
> o: ipaca
>
> Attaching relevant logs and CC-ing Ade, I think we will need his help on this one.
>
> IMO it is important to have this scenario working too, F17 IPA users who'd
> upgrade to F18 and try to setup a replica may hit the same issue.
>
> Martin
>

I have successfully installed a 3.1 replica of a 3.0 master on my VMs.

I believe the crash is due to broken setup, but I'll leave it to Ade to 
look at the Java exceptions.

-- 
Petr³