[Freeipa-users] Latest FreeIPA update causing problems

Thu Feb 16 20:39:57 UTC 2012

On 02/16/2012 01:12 PM, Dan Scott wrote:
> On Thu, Feb 16, 2012 at 14:24, Rich Megginson<rmeggins at redhat.com>  wrote:
>> On 02/16/2012 10:40 AM, Dan Scott wrote:
>>> Hi,
>>>
>>> On Thu, Feb 16, 2012 at 11:56, Rich Megginson<rmeggins at redhat.com>    wrote:
>>>> On 02/16/2012 09:12 AM, Dan Scott wrote:
>>>>> Hi,
>>>>>
>>>>> On Thu, Feb 16, 2012 at 10:37, Rich Megginson<rmeggins at redhat.com>
>>>>>   wrote:
>>>>>> On 02/16/2012 08:26 AM, Dan Scott wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> I have recently upgraded one of my FreeIPA servers (Fedora 16) with
>>>>>>> the latest package versions:
>>>>>>>
>>>>>>> Feb 15 14:10:19 Updated: libselinux-2.1.6-6.fc16.x86_64
>>>>>>> Feb 15 14:10:20 Updated: krb5-libs-1.9.2-6.fc16.x86_64
>>>>>>> Feb 15 14:10:21 Updated: systemd-37-13.fc16.x86_64
>>>>>>> Feb 15 14:10:22 Updated: systemd-units-37-13.fc16.x86_64
>>>>>>> Feb 15 14:10:22 Updated: device-mapper-libs-1.02.65-6.fc16.x86_64
>>>>>>> Feb 15 14:10:22 Updated: device-mapper-1.02.65-6.fc16.x86_64
>>>>>>> Feb 15 14:10:23 Updated: rpm-4.9.1.2-5.fc16.x86_64
>>>>>>> Feb 15 14:10:24 Updated: rpm-libs-4.9.1.2-5.fc16.x86_64
>>>>>>> Feb 15 14:10:24 Updated:
>>>>>>> device-mapper-event-libs-1.02.65-6.fc16.x86_64
>>>>>>> Feb 15 14:10:26 Updated: freeipa-python-2.1.4-5.fc16.x86_64
>>>>>>> Feb 15 14:10:26 Updated: systemd-sysv-37-13.fc16.x86_64
>>>>>>> Feb 15 14:10:27 Updated: krb5-server-1.9.2-6.fc16.x86_64
>>>>>>> Feb 15 14:10:27 Updated: krb5-server-ldap-1.9.2-6.fc16.x86_64
>>>>>>> Feb 15 14:10:27 Updated: device-mapper-event-1.02.65-6.fc16.x86_64
>>>>>>> Feb 15 14:10:28 Updated: lvm2-libs-2.02.86-6.fc16.x86_64
>>>>>>> Feb 15 14:10:28 Updated: rpm-build-libs-4.9.1.2-5.fc16.x86_64
>>>>>>> Feb 15 14:10:28 Updated: mod_auth_kerb-5.4-8.fc16.x86_64
>>>>>>> Feb 15 14:10:28 Updated: 389-ds-base-libs-1.2.10-0.10.rc1.fc16.x86_64
>>>>>>> Feb 15 14:10:30 Updated: 389-ds-base-1.2.10-0.10.rc1.fc16.x86_64
>>>>>>> Feb 15 14:10:31 Updated: krb5-pkinit-openssl-1.9.2-6.fc16.x86_64
>>>>>>> Feb 15 14:10:31 Updated: krb5-workstation-1.9.2-6.fc16.x86_64
>>>>>>> Feb 15 14:10:31 Updated: freeipa-client-2.1.4-5.fc16.x86_64
>>>>>>> Feb 15 14:10:31 Updated: freeipa-admintools-2.1.4-5.fc16.x86_64
>>>>>>> Feb 15 14:11:47 Updated: freeipa-server-2.1.4-5.fc16.x86_64
>>>>>>> Feb 15 14:15:19 Updated: freeipa-server-selinux-2.1.4-5.fc16.x86_64
>>>>>>> Feb 15 14:15:19 Updated: rpm-python-4.9.1.2-5.fc16.x86_64
>>>>>>> Feb 15 14:15:20 Updated: lvm2-2.02.86-6.fc16.x86_64
>>>>>>> Feb 15 14:15:20 Updated: libselinux-python-2.1.6-6.fc16.x86_64
>>>>>>> Feb 15 14:15:20 Updated: libselinux-utils-2.1.6-6.fc16.x86_64
>>>>>>> Feb 15 14:15:21 Updated: alsa-lib-1.0.25-1.fc16.x86_64
>>>>>>> Feb 15 14:15:30 Installed: kernel-3.2.6-3.fc16.x86_64
>>>>>>>
>>>>>>> I am having major problems with freeipa services (I replaced my real
>>>>>>> domain with example.com):
>>>>>>>
>>>>>>> [root at fileserver3 ~]# ipactl status
>>>>>>> Directory Service: STOPPED
>>>>>>> Unknown error when retrieving list of services from LDAP: [Errno 111]
>>>>>>> Connection refused
>>>>>>> [root at fileserver3 ~]# ipactl start
>>>>>>> Starting Directory Service
>>>>>>> Failed to read data from Directory Service: Failed to get list of
>>>>>>> services to probe status!
>>>>>>> Configured hostname 'fileserver3.example.com' does not match any
>>>>>>> master server in LDAP:
>>>>>>> No master found because of error: {'matched': 'dc=example,dc=com',
>>>>>>> 'desc': 'No such object'}
>>>>>>> Shutting down
>>>>>>> [root at fileserver3 ~]#
>>>>>>>
>>>>>>> None of the IPA processes will start. The dirsrv error log shows:
>>>>>>>
>>>>>>> [16/Feb/2012:10:20:23 -0500] - 389-Directory/1.2.10.rc1 B2012.035.328
>>>>>>> starting up
>>>>>>> [16/Feb/2012:10:20:23 -0500] schema-compat-plugin - warning: no
>>>>>>> entries set up under cn=groups, cn=compat,dc=example,dc=com
>>>>>>> [16/Feb/2012:10:20:23 -0500] schema-compat-plugin - warning: no
>>>>>>> entries set up under cn=ng, cn=compat,dc=example,dc=com
>>>>>>> [16/Feb/2012:10:20:23 -0500] schema-compat-plugin - warning: no
>>>>>>> entries set up under ou=sudoers,dc=example,dc=com
>>>>>>> [16/Feb/2012:10:20:23 -0500] schema-compat-plugin - warning: no
>>>>>>> entries set up under cn=users, cn=compat,dc=example,dc=com
>>>>>>> [16/Feb/2012:10:20:23 -0500] dna-plugin - dna_parse_config_entry:
>>>>>>> Unable to locate shared configuration entry
>>>>>>> (cn=posix-ids,cn=dna,cn=ipa,cn=etc,dc=example,dc=com)
>>>>>>> [16/Feb/2012:10:20:23 -0500] dna-plugin - dna_parse_config_entry:
>>>>>>> Invalid config entry [cn=posix ids,cn=distributed numeric assignment
>>>>>>> plugin,cn=plugins,cn=config] skipped
>>>>>>> [16/Feb/2012:10:20:23 -0500] - slapd started.  Listening on All
>>>>>>> Interfaces port 389 for LDAP requests
>>>>>>> [16/Feb/2012:10:20:23 -0500] - Listening on All Interfaces port 636
>>>>>>> for LDAPS requests
>>>>>>> [16/Feb/2012:10:20:23 -0500] - Listening on
>>>>>>> /var/run/slapd-EXAMPLE-COM.socket for LDAPI requests
>>>>>>> [16/Feb/2012:10:20:23 -0500] - slapd shutting down - signaling
>>>>>>> operation
>>>>>>> threads
>>>>>>> [16/Feb/2012:10:20:23 -0500] - slapd shutting down - closing down
>>>>>>> internal subsystems and plugins
>>>>>>> [16/Feb/2012:10:20:24 -0500] - Waiting for 4 database threads to stop
>>>>>>> [16/Feb/2012:10:20:24 -0500] - All database threads now stopped
>>>>>>> [16/Feb/2012:10:20:24 -0500] - slapd stopped.
>>>>>>>
>>>>>>> Can someone help?
>>>>>> start your directory server - systemctl start dirsrv.target
>>>>>> do a search for the dna entries:
>>>>>> ldapsearch -xLLL -D "cn=directory manager" -W -s one -b
>>>>>> "cn=dna,cn=ipa,cn=etc,dc=example,dc=com"
>>>>>>
>>>>>> and
>>>>>> ldapsearch -xLLL -D "cn=directory manager" -W -s one -b "cn=distributed
>>>>>> numeric assignment
>>>>>> plugin,cn=plugins,cn=config"
>>>>> Results:
>>>>>
>>>>> [root at fileserver3 ~]# ldapsearch -xLLL -D "cn=directory manager" -W -s
>>>>> one -b "cn=dna,cn=ipa,cn=etc,dc=example,dc=com"
>>>>> Enter LDAP Password:
>>>>> No such object (32)
>>>>> Matched DN: dc=example,dc=com
>>>>> [root at fileserver3 ~]# ldapsearch -xLLL -D "cn=directory manager" -W -s
>>>>> one -b "cn=distributed numeric assignment plugin,cn=plugins,cn=config"
>>>>> Enter LDAP Password:
>>>>> dn: cn=Posix IDs,cn=Distributed Numeric Assignment
>>>>> Plugin,cn=plugins,cn=config
>>>>> objectClass: top
>>>>> objectClass: extensibleObject
>>>>> cn: Posix IDs
>>>>> dnatype: uidNumber
>>>>> dnatype: gidNumber
>>>>> dnanextvalue: 1101
>>>>> dnamaxvalue: 1100
>>>>> dnamagicregen: 999
>>>>> dnafilter: (|(objectclass=posixAccount)(objectClass=posixGroup))
>>>>> dnascope: dc=example,dc=com
>>>>> dnathreshold: 500
>>>>> dnasharedcfgdn: cn=posix-ids,cn=dna,cn=ipa,cn=etc,dc=example,dc=com
>>>>>
>>>>> It looks like all my data is missing.... do I need to re-initialize
>>>>> the replication?
>>>> Is this your master or a replica?
>>>> You can look at the database directly with
>>>> dbscan -f /var/lib/dirsrv/slapd-DOMAIN/db/userRoot/id2entry.db4
>>>> you can also export it to ldif with
>>>> /var/lib/dirsrv/scripts-DOMAIN/db2ldif -n userRoot -a
>>>> /var/lib/dirsrv/slapd-DOMAIN/ldif/mydb.ldif
>>> It's a replica. Luckily the master hasn't been updated yet. I have
>>> another replica running Fedora 15 which seems OK as well.
>>>
>>> The dbscan command looks good, I think. I can see an entry for "rdn:
>>> uid=djscott".
>>>
>>> I ran the export, and got:
>>>
>>> Exported ldif file: /var/lib/dirsrv/slapd-DOMAIN/ldif/mydb.ldif
>>> ldiffile: /var/lib/dirsrv/slapd-DOMAIN/ldif/mydb.ldif
>>> [16/Feb/2012:12:37:40 -0500] - export userRoot: Processed 437 entries
>>> (100%).
>>> [16/Feb/2012:12:37:40 -0500] - All database threads now stopped
>>>
>>> The ldif file looks good, thanks. Nice to know that the data is all
>>> still there. Any ideas why it's not showing up when I query LDAP?
>> So you do see an entry for
>> cn=posix-ids,cn=dna,cn=ipa,cn=etc,dc=example,dc=com in your dbscan output
>> and in the mydb.ldif file?
>> The dbscan output should contain an entry ID and a parent entry ID - this
>> will be a one, two, or three digit integer.
>> try the following, where X is the entry ID, and Y is the parent entry ID:
>> dbscan -k X -f /var/lib/dirsrv/slapd-DOMAIN/db/userRoot/entryrdn.db4
>> dbscan -k Y -f /var/lib/dirsrv/slapd-DOMAIN/db/userRoot/entryrdn.db4
>> dbscan -k PX -f /var/lib/dirsrv/slapd-DOMAIN/db/userRoot/entryrdn.db4
>> dbscan -k CY -f /var/lib/dirsrv/slapd-DOMAIN/db/userRoot/entryrdn.db4
> Yep, there's an entry for posix-ids, and an entry for each of my
> replica servers (I only show 1 here, but there are others):
>
> # entry-id: 29
> dn: cn=posix-ids,cn=dna,cn=ipa,cn=etc,dc=example,dc=com
> nsUniqueId: 4fff5921-e48611e0-bf3681aa-d1a3957d
> modifyTimestamp: 20110921191715Z
> createTimestamp: 20110921191715Z
> modifiersName: cn=directory manager
> creatorsName: cn=directory manager
> cn: posix-ids
> objectClass: nsContainer
> objectClass: top
>
> # entry-id: 446
> dn: dnaHostname=fileserver3.example.com+dnaPortNum=389,cn=posix-ids,cn=dna,cn=
>   ipa,cn=etc,dc=example,dc=com
> nsUniqueId: 47743a07-57fc11e1-b1edce26-60f19ec1
> objectClass: extensibleObject
> objectClass: top
> dnahostname: fileserver3.example.com
> dnaportnum: 389
> dnasecureportnum: 636
> dnaremainingvalues: 0
> creatorsName: cn=Distributed Numeric Assignment Plugin,cn=plugins,cn=config
> modifiersName: cn=Distributed Numeric Assignment Plugin,cn=plugins,cn=config
> createTimestamp: 20120215174154Z
> modifyTimestamp: 20120215174154Z
>
> The:
>
> dbscan -k 29 .....
>
> Gives:
>
> "Can't find key '29'".
>
> But from the manpage, you maybe mean 'K'? Even so, it still doesn't
> look good with either -K 29 or -K P29.
It is confusing - for the id2entry.db4 file, and only for that file, you 
use -K - for the entryrdn.db4 and the other index files, you use -k
> Can't set cursor to returned item: DB_NOTFOUND: No matching key/data pair found
>
> dbscan -r shows (with some manual grepping):
>
> 29:cn=posix-ids
>    ID: 29; RDN: "cn=posix-ids"; NRDN: "cn=posix-ids"
>
> P29:cn=posix-ids
>    ID: 28; RDN: "cn=dna"; NRDN: "cn=dna"
>
> Does this mean that the index is OK but the data is missing? I'm not
> really sure what we're looking for here.
The entryrdn index was supposed to be upgraded during the yum update to 
1.2.10.rc1.  It looks as though that did not happen.  The format should be
29
   ID: 29; RDN: "cn=posix-ids"; NRDN: "cn=posix-ids"

The fact that there is still the ":<rdn>" in there means the upgrade 
didn't work.

Do you have your errors log from around the time you did the yum upgrade 
that upgraded 389-ds-base to 1.2.10.rc1?
> Does LDAP have indexes
> similar to the way that RDBMSs do?
Yes.
> Thanks,