[Freeipa-users] ipa migrate-ds failing when more than 1 namingcontext is available

Fri Jan 27 17:44:58 UTC 2012

On 01/27/2012 06:15 PM, Sigbjorn Lie wrote:
> On 01/27/2012 03:55 PM, Rob Crittenden wrote:
>> Sigbjorn Lie wrote:
>>>
>>> On Fri, January 27, 2012 15:37, Rob Crittenden wrote:
>>>> Stephen Gallagher wrote:
>>>>
>>>>> On Fri, 2012-01-27 at 15:11 +0100, Sigbjorn Lie wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>>
>>>>>> The first naming context returned from the LDAP server is always 
>>>>>> chosen
>>>>>> when using migrate-ds. This makes my import fail when I attempt 
>>>>>> to import users and groups from
>>>>>> a previous LDAP server having more than 1 naming contexts available.
>>>>>>
>>>>>> The migrate-ds script should accept an option to specify what 
>>>>>> base_dn I
>>>>>> would like to import from.
>>>>>>
>>>>>> Is there such an option today? I cannot find it...
>>>>>>
>>>>
>>>> Not currently. I noticed this earlier in the week and opened a 
>>>> ticket on
>>>> it, https://fedorahosted.org/freeipa/ticket/2314
>>>>
>>>>>
>>>>> Just to add to this request, if the original LDAP server has a
>>>>> defaultNamingContext attribute, it should be honored for 
>>>>> auto-detecting which base to migrate.
>>>>
>>>> I'll update the 2314 to ensure we don't forget about this. 389-ds just
>>>> added support for defaultNamingContext.
>>>>
>>>
>>> Ok, thank you.
>>>
>>> Anything I can do to work around this issue today? I suppose there 
>>> is just a file that need to be
>>> hacked to set a set a value instead of the auto-detected value... ?
>>>
>>
>> /usr/lib/python*/site-packages/ipalib/plugins/migration.py
>>
>> ~line 620 you'll see a block starting with the comment "retrieve DS 
>> base DN".
>>
>> Comment out the next 8 lines by prefixing them with # (these query to 
>> get the namingContext then pull the first value out).
>>
>> Add:
>>
>> ds_base_dn = 'dc=yourbasedn,dc=com'
>>
>> Alternatively you could always just add the above line to override 
>> what is detected. Commenting out just saves an LDAP lookup.
>>
>> Restart Apache.
>
>
> I already found that file and did that earlier today, however I was 
> restarting tomcat6, not httpd... my bad. :)
>
> I have to specify --group-objectclass=posixGroup to get groups 
> imported, that's fine. But I only get a few users imported. I see that 
> by default it seem to be looking for objectclass=person. Only a few 
> user accounts have that objectclass associated, so I add 
> --user-objectclass=posixAccount as all users have this objectclass 
> associated with their account.
>
> $ ipa migrate-ds --user-container='ou=people' 
> --group-container='ou=group' --bind-dn='cn=directory manager' 
> --user-objectclass=account --group-objectclass=posixGroup 
> --schema=RFC2307 --continue ldap://ldapserver:399
> ipa: ERROR: an internal error has occurred
>
> Not good. I look in the /var/log/httpd/error_log file, and I find:
>
> [Fri Jan 27 18:12:51 2012] [error] ipa: INFO: admin at NONE: ping(): SUCCESS
> [Fri Jan 27 18:12:52 2012] [error] ipa: ERROR: non-public: 
> UnicodeDecodeError: 'utf8' codec can't decode byte 0xe5 in position 1: 
> invalid continuation byte
> [Fri Jan 27 18:12:52 2012] [error] Traceback (most recent call last):
> [Fri Jan 27 18:12:52 2012] [error]   File 
> "/usr/lib/python2.6/site-packages/ipaserver/rpcserver.py", line 228, 
> in wsgi_execute
> [Fri Jan 27 18:12:52 2012] [error]     result = 
> self.Command[name](*args, **options)
> [Fri Jan 27 18:12:52 2012] [error]   File 
> "/usr/lib/python2.6/site-packages/ipalib/frontend.py", line 432, in 
> __call__
> [Fri Jan 27 18:12:52 2012] [error]     ret = self.run(*args, **options)
> [Fri Jan 27 18:12:52 2012] [error]   File 
> "/usr/lib/python2.6/site-packages/ipalib/frontend.py", line 738, in run
> [Fri Jan 27 18:12:52 2012] [error]     return self.execute(*args, 
> **options)
> [Fri Jan 27 18:12:52 2012] [error]   File 
> "/usr/lib/python2.6/site-packages/ipalib/plugins/migration.py", line 
> 634, in execute
> [Fri Jan 27 18:12:52 2012] [error]     ldap, config, ds_ldap, 
> ds_base_dn, options
> [Fri Jan 27 18:12:52 2012] [error]   File 
> "/usr/lib/python2.6/site-packages/ipalib/plugins/migration.py", line 
> 513, in migrate
> [Fri Jan 27 18:12:52 2012] [error]     search_refs=True    # migrated 
> DS may contain search references
> [Fri Jan 27 18:12:52 2012] [error]   File 
> "/usr/lib/python2.6/site-packages/ipalib/encoder.py", line 188, in new_f
> [Fri Jan 27 18:12:52 2012] [error]     return f(*new_args, **kwargs)
> [Fri Jan 27 18:12:52 2012] [error]   File 
> "/usr/lib/python2.6/site-packages/ipalib/encoder.py", line 199, in new_f
> [Fri Jan 27 18:12:52 2012] [error]     return args[0].decode(f(*args, 
> **kwargs))
> [Fri Jan 27 18:12:52 2012] [error]   File 
> "/usr/lib/python2.6/site-packages/ipalib/encoder.py", line 139, in decode
> [Fri Jan 27 18:12:52 2012] [error]     return tuple(self.decode(m) for 
> m in var)
> [Fri Jan 27 18:12:52 2012] [error]   File 
> "/usr/lib/python2.6/site-packages/ipalib/encoder.py", line 139, in 
> <genexpr>
> [Fri Jan 27 18:12:52 2012] [error]     return tuple(self.decode(m) for 
> m in var)
> [Fri Jan 27 18:12:52 2012] [error]   File 
> "/usr/lib/python2.6/site-packages/ipalib/encoder.py", line 137, in decode
> [Fri Jan 27 18:12:52 2012] [error]     return [self.decode(m) for m in 
> var]
> [Fri Jan 27 18:12:52 2012] [error]   File 
> "/usr/lib/python2.6/site-packages/ipalib/encoder.py", line 139, in decode
> [Fri Jan 27 18:12:52 2012] [error]     return tuple(self.decode(m) for 
> m in var)
> [Fri Jan 27 18:12:52 2012] [error]   File 
> "/usr/lib/python2.6/site-packages/ipalib/encoder.py", line 139, in 
> <genexpr>
> [Fri Jan 27 18:12:52 2012] [error]     return tuple(self.decode(m) for 
> m in var)
> [Fri Jan 27 18:12:52 2012] [error]   File 
> "/usr/lib/python2.6/site-packages/ipalib/encoder.py", line 157, in decode
> [Fri Jan 27 18:12:52 2012] [error]     dct[k] = 
> self._decode_dict_val(k, v)
> [Fri Jan 27 18:12:52 2012] [error]   File 
> "/usr/lib/python2.6/site-packages/ipalib/encoder.py", line 64, in 
> _decode_dict_val
> [Fri Jan 27 18:12:52 2012] [error]     return self.decode(val)
> [Fri Jan 27 18:12:52 2012] [error]   File 
> "/usr/lib/python2.6/site-packages/ipalib/encoder.py", line 137, in decode
> [Fri Jan 27 18:12:52 2012] [error]     return [self.decode(m) for m in 
> var]
> [Fri Jan 27 18:12:52 2012] [error]   File 
> "/usr/lib/python2.6/site-packages/ipalib/encoder.py", line 132, in decode
> [Fri Jan 27 18:12:52 2012] [error]     
> var.decode(self.encoder_settings.decode_from)
> [Fri Jan 27 18:12:52 2012] [error]   File 
> "/usr/lib64/python2.6/encodings/utf_8.py", line 16, in decode
> [Fri Jan 27 18:12:52 2012] [error]     return 
> codecs.utf_8_decode(input, errors, True)
> [Fri Jan 27 18:12:52 2012] [error] UnicodeDecodeError: 'utf8' codec 
> can't decode byte 0xe5 in position 1: invalid continuation byte
> [Fri Jan 27 18:12:52 2012] [error] ipa: INFO: admin at NONE: 
> migrate_ds(u'ldap://svg-p-idm02.none:389', u'********', 
> binddn=u'cn=directory manager', usercontainer=u'ou=people', 
> groupcontainer=u'ou=group', userobjectclass=(u'account',), 
> groupobjectclass=(u'posixGroup',), userignoreobjectclass=None, 
> userignoreattribute=None, groupignoreobjectclass=None, 
> groupignoreattribute=None, groupoverwritegid=False, schema=u'RFC2307', 
> continue=False, exclude_groups=None, exclude_users=None): 
> UnicodeDecodeError
>
>
> Any suggestions?
>
>
Oh yes and of course I've already looked for accounts with any non-utf8 
chars in any of the output of an ldapsearch of the same ldap tree I'm 
trying to import from...