[Freeipa-users] Latest FreeIPA update causing problems

[Dan Scott]

On Thu, Feb 16, 2012 at 14:24, Rich Megginson <rmeggins at redhat.com> wrote:
> On 02/16/2012 10:40 AM, Dan Scott wrote:
>>
>> Hi,
>>
>> On Thu, Feb 16, 2012 at 11:56, Rich Megginson<rmeggins at redhat.com>  wrote:
>>>
>>> On 02/16/2012 09:12 AM, Dan Scott wrote:
>>>>
>>>> Hi,
>>>>
>>>> On Thu, Feb 16, 2012 at 10:37, Rich Megginson<rmeggins at redhat.com>
>>>>  wrote:
>>>>>
>>>>> On 02/16/2012 08:26 AM, Dan Scott wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I have recently upgraded one of my FreeIPA servers (Fedora 16) with
>>>>>> the latest package versions:
>>>>>>
>>>>>> Feb 15 14:10:19 Updated: libselinux-2.1.6-6.fc16.x86_64
>>>>>> Feb 15 14:10:20 Updated: krb5-libs-1.9.2-6.fc16.x86_64
>>>>>> Feb 15 14:10:21 Updated: systemd-37-13.fc16.x86_64
>>>>>> Feb 15 14:10:22 Updated: systemd-units-37-13.fc16.x86_64
>>>>>> Feb 15 14:10:22 Updated: device-mapper-libs-1.02.65-6.fc16.x86_64
>>>>>> Feb 15 14:10:22 Updated: device-mapper-1.02.65-6.fc16.x86_64
>>>>>> Feb 15 14:10:23 Updated: rpm-4.9.1.2-5.fc16.x86_64
>>>>>> Feb 15 14:10:24 Updated: rpm-libs-4.9.1.2-5.fc16.x86_64
>>>>>> Feb 15 14:10:24 Updated:
>>>>>> device-mapper-event-libs-1.02.65-6.fc16.x86_64
>>>>>> Feb 15 14:10:26 Updated: freeipa-python-2.1.4-5.fc16.x86_64
>>>>>> Feb 15 14:10:26 Updated: systemd-sysv-37-13.fc16.x86_64
>>>>>> Feb 15 14:10:27 Updated: krb5-server-1.9.2-6.fc16.x86_64
>>>>>> Feb 15 14:10:27 Updated: krb5-server-ldap-1.9.2-6.fc16.x86_64
>>>>>> Feb 15 14:10:27 Updated: device-mapper-event-1.02.65-6.fc16.x86_64
>>>>>> Feb 15 14:10:28 Updated: lvm2-libs-2.02.86-6.fc16.x86_64
>>>>>> Feb 15 14:10:28 Updated: rpm-build-libs-4.9.1.2-5.fc16.x86_64
>>>>>> Feb 15 14:10:28 Updated: mod_auth_kerb-5.4-8.fc16.x86_64
>>>>>> Feb 15 14:10:28 Updated: 389-ds-base-libs-1.2.10-0.10.rc1.fc16.x86_64
>>>>>> Feb 15 14:10:30 Updated: 389-ds-base-1.2.10-0.10.rc1.fc16.x86_64
>>>>>> Feb 15 14:10:31 Updated: krb5-pkinit-openssl-1.9.2-6.fc16.x86_64
>>>>>> Feb 15 14:10:31 Updated: krb5-workstation-1.9.2-6.fc16.x86_64
>>>>>> Feb 15 14:10:31 Updated: freeipa-client-2.1.4-5.fc16.x86_64
>>>>>> Feb 15 14:10:31 Updated: freeipa-admintools-2.1.4-5.fc16.x86_64
>>>>>> Feb 15 14:11:47 Updated: freeipa-server-2.1.4-5.fc16.x86_64
>>>>>> Feb 15 14:15:19 Updated: freeipa-server-selinux-2.1.4-5.fc16.x86_64
>>>>>> Feb 15 14:15:19 Updated: rpm-python-4.9.1.2-5.fc16.x86_64
>>>>>> Feb 15 14:15:20 Updated: lvm2-2.02.86-6.fc16.x86_64
>>>>>> Feb 15 14:15:20 Updated: libselinux-python-2.1.6-6.fc16.x86_64
>>>>>> Feb 15 14:15:20 Updated: libselinux-utils-2.1.6-6.fc16.x86_64
>>>>>> Feb 15 14:15:21 Updated: alsa-lib-1.0.25-1.fc16.x86_64
>>>>>> Feb 15 14:15:30 Installed: kernel-3.2.6-3.fc16.x86_64
>>>>>>
>>>>>> I am having major problems with freeipa services (I replaced my real
>>>>>> domain with example.com):
>>>>>>
>>>>>> [root at fileserver3 ~]# ipactl status
>>>>>> Directory Service: STOPPED
>>>>>> Unknown error when retrieving list of services from LDAP: [Errno 111]
>>>>>> Connection refused
>>>>>> [root at fileserver3 ~]# ipactl start
>>>>>> Starting Directory Service
>>>>>> Failed to read data from Directory Service: Failed to get list of
>>>>>> services to probe status!
>>>>>> Configured hostname 'fileserver3.example.com' does not match any
>>>>>> master server in LDAP:
>>>>>> No master found because of error: {'matched': 'dc=example,dc=com',
>>>>>> 'desc': 'No such object'}
>>>>>> Shutting down
>>>>>> [root at fileserver3 ~]#
>>>>>>
>>>>>> None of the IPA processes will start. The dirsrv error log shows:
>>>>>>
>>>>>> [16/Feb/2012:10:20:23 -0500] - 389-Directory/1.2.10.rc1 B2012.035.328
>>>>>> starting up
>>>>>> [16/Feb/2012:10:20:23 -0500] schema-compat-plugin - warning: no
>>>>>> entries set up under cn=groups, cn=compat,dc=example,dc=com
>>>>>> [16/Feb/2012:10:20:23 -0500] schema-compat-plugin - warning: no
>>>>>> entries set up under cn=ng, cn=compat,dc=example,dc=com
>>>>>> [16/Feb/2012:10:20:23 -0500] schema-compat-plugin - warning: no
>>>>>> entries set up under ou=sudoers,dc=example,dc=com
>>>>>> [16/Feb/2012:10:20:23 -0500] schema-compat-plugin - warning: no
>>>>>> entries set up under cn=users, cn=compat,dc=example,dc=com
>>>>>> [16/Feb/2012:10:20:23 -0500] dna-plugin - dna_parse_config_entry:
>>>>>> Unable to locate shared configuration entry
>>>>>> (cn=posix-ids,cn=dna,cn=ipa,cn=etc,dc=example,dc=com)
>>>>>> [16/Feb/2012:10:20:23 -0500] dna-plugin - dna_parse_config_entry:
>>>>>> Invalid config entry [cn=posix ids,cn=distributed numeric assignment
>>>>>> plugin,cn=plugins,cn=config] skipped
>>>>>> [16/Feb/2012:10:20:23 -0500] - slapd started.  Listening on All
>>>>>> Interfaces port 389 for LDAP requests
>>>>>> [16/Feb/2012:10:20:23 -0500] - Listening on All Interfaces port 636
>>>>>> for LDAPS requests
>>>>>> [16/Feb/2012:10:20:23 -0500] - Listening on
>>>>>> /var/run/slapd-EXAMPLE-COM.socket for LDAPI requests
>>>>>> [16/Feb/2012:10:20:23 -0500] - slapd shutting down - signaling
>>>>>> operation
>>>>>> threads
>>>>>> [16/Feb/2012:10:20:23 -0500] - slapd shutting down - closing down
>>>>>> internal subsystems and plugins
>>>>>> [16/Feb/2012:10:20:24 -0500] - Waiting for 4 database threads to stop
>>>>>> [16/Feb/2012:10:20:24 -0500] - All database threads now stopped
>>>>>> [16/Feb/2012:10:20:24 -0500] - slapd stopped.
>>>>>>
>>>>>> Can someone help?
>>>>>
>>>>> start your directory server - systemctl start dirsrv.target
>>>>> do a search for the dna entries:
>>>>> ldapsearch -xLLL -D "cn=directory manager" -W -s one -b
>>>>> "cn=dna,cn=ipa,cn=etc,dc=example,dc=com"
>>>>>
>>>>> and
>>>>> ldapsearch -xLLL -D "cn=directory manager" -W -s one -b "cn=distributed
>>>>> numeric assignment
>>>>> plugin,cn=plugins,cn=config"
>>>>
>>>> Results:
>>>>
>>>> [root at fileserver3 ~]# ldapsearch -xLLL -D "cn=directory manager" -W -s
>>>> one -b "cn=dna,cn=ipa,cn=etc,dc=example,dc=com"
>>>> Enter LDAP Password:
>>>> No such object (32)
>>>> Matched DN: dc=example,dc=com
>>>> [root at fileserver3 ~]# ldapsearch -xLLL -D "cn=directory manager" -W -s
>>>> one -b "cn=distributed numeric assignment plugin,cn=plugins,cn=config"
>>>> Enter LDAP Password:
>>>> dn: cn=Posix IDs,cn=Distributed Numeric Assignment
>>>> Plugin,cn=plugins,cn=config
>>>> objectClass: top
>>>> objectClass: extensibleObject
>>>> cn: Posix IDs
>>>> dnatype: uidNumber
>>>> dnatype: gidNumber
>>>> dnanextvalue: 1101
>>>> dnamaxvalue: 1100
>>>> dnamagicregen: 999
>>>> dnafilter: (|(objectclass=posixAccount)(objectClass=posixGroup))
>>>> dnascope: dc=example,dc=com
>>>> dnathreshold: 500
>>>> dnasharedcfgdn: cn=posix-ids,cn=dna,cn=ipa,cn=etc,dc=example,dc=com
>>>>
>>>> It looks like all my data is missing.... do I need to re-initialize
>>>> the replication?
>>>
>>> Is this your master or a replica?
>>> You can look at the database directly with
>>> dbscan -f /var/lib/dirsrv/slapd-DOMAIN/db/userRoot/id2entry.db4
>>> you can also export it to ldif with
>>> /var/lib/dirsrv/scripts-DOMAIN/db2ldif -n userRoot -a
>>> /var/lib/dirsrv/slapd-DOMAIN/ldif/mydb.ldif
>>
>> It's a replica. Luckily the master hasn't been updated yet. I have
>> another replica running Fedora 15 which seems OK as well.
>>
>> The dbscan command looks good, I think. I can see an entry for "rdn:
>> uid=djscott".
>>
>> I ran the export, and got:
>>
>> Exported ldif file: /var/lib/dirsrv/slapd-DOMAIN/ldif/mydb.ldif
>> ldiffile: /var/lib/dirsrv/slapd-DOMAIN/ldif/mydb.ldif
>> [16/Feb/2012:12:37:40 -0500] - export userRoot: Processed 437 entries
>> (100%).
>> [16/Feb/2012:12:37:40 -0500] - All database threads now stopped
>>
>> The ldif file looks good, thanks. Nice to know that the data is all
>> still there. Any ideas why it's not showing up when I query LDAP?
>
> So you do see an entry for
> cn=posix-ids,cn=dna,cn=ipa,cn=etc,dc=example,dc=com in your dbscan output
> and in the mydb.ldif file?
> The dbscan output should contain an entry ID and a parent entry ID - this
> will be a one, two, or three digit integer.
> try the following, where X is the entry ID, and Y is the parent entry ID:
> dbscan -k X -f /var/lib/dirsrv/slapd-DOMAIN/db/userRoot/entryrdn.db4
> dbscan -k Y -f /var/lib/dirsrv/slapd-DOMAIN/db/userRoot/entryrdn.db4
> dbscan -k PX -f /var/lib/dirsrv/slapd-DOMAIN/db/userRoot/entryrdn.db4
> dbscan -k CY -f /var/lib/dirsrv/slapd-DOMAIN/db/userRoot/entryrdn.db4

Yep, there's an entry for posix-ids, and an entry for each of my
replica servers (I only show 1 here, but there are others):

# entry-id: 29
dn: cn=posix-ids,cn=dna,cn=ipa,cn=etc,dc=example,dc=com
nsUniqueId: 4fff5921-e48611e0-bf3681aa-d1a3957d
modifyTimestamp: 20110921191715Z
createTimestamp: 20110921191715Z
modifiersName: cn=directory manager
creatorsName: cn=directory manager
cn: posix-ids
objectClass: nsContainer
objectClass: top

# entry-id: 446
dn: dnaHostname=fileserver3.example.com+dnaPortNum=389,cn=posix-ids,cn=dna,cn=
 ipa,cn=etc,dc=example,dc=com
nsUniqueId: 47743a07-57fc11e1-b1edce26-60f19ec1
objectClass: extensibleObject
objectClass: top
dnahostname: fileserver3.example.com
dnaportnum: 389
dnasecureportnum: 636
dnaremainingvalues: 0
creatorsName: cn=Distributed Numeric Assignment Plugin,cn=plugins,cn=config
modifiersName: cn=Distributed Numeric Assignment Plugin,cn=plugins,cn=config
createTimestamp: 20120215174154Z
modifyTimestamp: 20120215174154Z

The:

dbscan -k 29 .....

Gives:

"Can't find key '29'".

But from the manpage, you maybe mean 'K'? Even so, it still doesn't
look good with either -K 29 or -K P29.

Can't set cursor to returned item: DB_NOTFOUND: No matching key/data pair found

dbscan -r shows (with some manual grepping):

29:cn=posix-ids
  ID: 29; RDN: "cn=posix-ids"; NRDN: "cn=posix-ids"

P29:cn=posix-ids
  ID: 28; RDN: "cn=dna"; NRDN: "cn=dna"

Does this mean that the index is OK but the data is missing? I'm not
really sure what we're looking for here. Does LDAP have indexes
similar to the way that RDBMSs do?

Thanks,