[Freeipa-users] migrate-ds aborts

[Martin Kosek]

On 01/16/2015 09:14 AM, Ludwig Krispenz wrote:
>
> On 01/16/2015 08:43 AM, Martin Kosek wrote:
>> On 01/15/2015 06:31 PM, Quayle, Bill wrote:
>>> I am migrating an openLDAP tree into ipa, and when I run ipa migrate-ds, the
>>> migration aborts after roughly 36 seconds with:
>>>
>>> ipa: ERROR: cannot connect to 'ldap://10.x.x.x:389’:
>>>
>>> It has transferred 9762 records, but seems to hit a timeout that causes it
>>> to stop.
>>>
>>> I’ve run it in debug mode, which only provides this:
>>>
>>> ipa: DEBUG: Starting external process
>>>
>>> ipa: DEBUG: args=keyctl pupdate 774698354
>>>
>>> ipa: DEBUG: Process finished, return code=0
>>>
>>> ipa: DEBUG: stdout=
>>>
>>> ipa: DEBUG: stderr=
>>>
>>> ipa: DEBUG: Caught fault 907 from server
>>> https://foo.example.com/ipa/session/xml: cannot connect to
>>> 'ldap://10.x.x.x:389':
>>>
>>> ipa: DEBUG: Destroyed connection context.xmlclient
>>>
>>> ipa: ERROR: cannot connect to 'ldap://10.x.x.x:389':
>>>
>>> Initially, it had transferred 2000 records and stopped, until I set
>>> nsslapd-sizelimit in cn=config:
>>>
>>> nsslapd-sizelimit: 20000
>>>
>>> I then re-ran the migration a dozen times, each time it would transfer more
>>> records, but would always time out at around the 36 second mark.  Now that I’m
>>> at 9762 records, it seems to have reached a peak.
>>>
>>> I suspect this is another tunable, but haven’t been able to find it, any
>>> document that mentions it, or anyone else hitting this issue.
>>>
>>> RHEL 7.0 server
>>>
>>> idM ipa-server-3.3.3-28
>>>
>>> source is RHEL 6.5 running openldap-2.4.23-34
>>>
>>> command used to migrate:
>>>
>>> ipa migrate-ds --continue --bind-dn="uid=me,ou=people,ou=foo,dc=example,dc=com"
>>> --base-dn="ou=foo,dc=example,dc=com" ldap://10.x.x.x:389
>>>
>>> *Cheers,*
>>>
>>> *-Bill*
>>
>> Ludwig, do you know? I am just thinking it may be also caused by some form of
>> timelimit, as mentioned in
>>
>> https://access.redhat.com/documentation/en-US/Red_Hat_Directory_Server/8.2/html/Administration_Guide/User_Account_Management-Setting_Resource_Limits_Based_on_the_Bind_DN.html
>>
>>
>> (those apply both for bind DNs and global cn=config). Maybe nsslapd-timelimit
>> could be increased? Although I saw the default is 3600, I assume it means 1
>> hour, i.e. not being the root cause.
> we need the access and error logs from DS, if it is a DS limit it should be
> seen in the err code.

+1

> Could it be that migrate-ds has it's own limit waiting for a repsponse from DS ?

The search itself in migrate-ds is limit-less:

             try:
                 entries, truncated = ds_ldap.find_entries(
                     search_filter, ['*'], search_bases[ldap_obj_name],
                     ds_ldap.SCOPE_ONELEVEL,
                     time_limit=0, size_limit=-1,
                     search_refs=True    # migrated DS may contain search 
references
                 )
              except...

Bill, I am wondering, could you add debug=True to /etc/ipa/default.conf on your 
server, reload the httpd process and re-run the migration? It should print 
additional debugging information that may help us.

Martin