[Freeipa-users] Stock with a Master in read-only mode - SOLVED

Martin Kosek mkosek at redhat.com
Tue May 27 13:15:39 UTC 2014 

On 05/27/2014 01:12 PM, Martin Kosek wrote:
> On 05/26/2014 09:00 PM, Davis Goodman wrote:
>> On Mon, May 26, 2014 at 1:17 PM, Davis Goodman <
>> davis.goodman at digital-district.ca> wrote:
>>
>>>
>>>
>>>
>>> On Mon, May 26, 2014 at 4:22 AM, Martin Kosek <mkosek at redhat.com> wrote:
>>>
>>>> On 05/25/2014 09:44 PM, Davis Goodman wrote:
>>>>> On Wed, May 21, 2014 at 12:06 PM, Martin Kosek <mkosek at redhat.com>
>>>> wrote:
>>>>>
>>>>>> On 05/21/2014 01:31 PM, Davis Goodman wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> <http://www.digital-district.ca/>
>>>>>>>
>>>>>>> On May 21, 2014, at 6:54 , Martin Kosek <mkosek at redhat.com
>>>>>>> <mailto:mkosek at redhat.com>> wrote:
>>>>>>>
>>>>>>>> On 05/21/2014 09:12 AM, Davis Goodman wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On May 21, 2014, at 2:45 , Martin Kosek <mkosek at redhat.com
>>>>>>>>> <mailto:mkosek at redhat.com>> wrote:
>>>>>>>>>
>>>>>>>>>> On 05/21/2014 08:36 AM, Davis Goodman wrote:
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Lately I’ve been having issues of replication between my server
>>>> and
>>>>>> my 2
>>>>>>>>>>> replicas.
>>>>>>>>>>>
>>>>>>>>>>> I decided I was going to delete my 2 replicas and start over
>>>> keeping
>>>>>> my
>>>>>>>>>>> master intact.
>>>>>>>>>>>
>>>>>>>>>>> I wasn`t successfull in getting all 3 servers to replicate to each
>>>>>> other. (
>>>>>>>>>>> it used to work)
>>>>>>>>>>>
>>>>>>>>>>> I tried deleting  1 replica after the other one  to always keep
>>>> one
>>>>>> of the
>>>>>>>>>>> two available.
>>>>>>>>>>>
>>>>>>>>>>> I had to delete manually the replica host on the master with a
>>>> bunch
>>>>>> of
>>>>>>>>>>> ldapdelete command which worked fine.
>>>>>>>>>>>
>>>>>>>>>>> But after many unsuccessful trials of getting everyone to sync I
>>>>>> decided to
>>>>>>>>>>> delete my two replicas.
>>>>>>>>>>>
>>>>>>>>>>> I went back to my master to use the ldapdelete to remove both
>>>> host`s
>>>>>>>>>>> records so that I could start over.
>>>>>>>>>>>
>>>>>>>>>>> Unfortunately now I’m getting this error.
>>>>>>>>>>>
>>>>>>>>>>> ldapdelete -x -D "cn=Directory Manager" -W
>>>>>>>>>>>   cn=DNS,cn=freeipa02.mtl.domain.int
>>>>>> ,cn=masters,cn=ipa,cn=etc,dc=domain,dc=int
>>>>>>>>>>> Enter LDAP Password:
>>>>>>>>>>> ldap_delete: Server is unwilling to perform (53)
>>>>>>>>>>> additional info: database is read-only
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I’m kinda stuck now with no replicas and no DNS. I could restore
>>>> the
>>>>>> backup
>>>>>>>>>>> prior to the start of the operation but with a master in read-only
>>>>>> mode it
>>>>>>>>>>> wouldn’t of much help.
>>>>>>>>>>>
>>>>>>>>>>> Any insights would be more than welcome.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Davis
>>>>>>>>>>
>>>>>>>>>> Hi Davis, did maybe some of your ipa-replica-manage crashed in a
>>>>>> middle of an
>>>>>>>>>> operation or an upgrade was interrupted  and left the database put
>>>> in
>>>>>> read only
>>>>>>>>>> mode?
>>>>>>>>>>
>>>>>>>>>> You can find out with this ldapsearch:
>>>>>>>>>>
>>>>>>>>>> ldapsearch -h `hostname` -D "cn=Directory Manager" -x -w kokos123
>>>> -b
>>>>>>>>>> 'cn=userRoot,cn=ldbm database,cn=plugins,cn=config' -s base
>>>>>>>>>>
>>>>>>>>>> Check for nsslapd-readonly, it should be put to "off" in normal
>>>>>> operation.
>>>>>>>>>>
>>>>>>>>>> Martin
>>>>>>>>> Ok finally managed to modify the read-only flag.
>>>>>>>>>
>>>>>>>>> Could prepare my replicas and get them going.
>>>>>>>>>
>>>>>>>>> Everything seems fine but I’m getting this error while setting up
>>>> the
>>>>>>>>> replicas. Should I be concerned about this one:
>>>>>>>>>
>>>>>>>>> Update in progress
>>>>>>>>> Update in progress
>>>>>>>>> Update in progress
>>>>>>>>> Update in progress
>>>>>>>>> Update in progress
>>>>>>>>> Update in progress
>>>>>>>>> Update succeeded
>>>>>>>>>  [23/31]: adding replication acis
>>>>>>>>>  [24/31]: setting Auto Member configuration
>>>>>>>>>  [25/31]: enabling S4U2Proxy delegation
>>>>>>>>> ipa         : CRITICAL Failed to load replica-s4u2proxy.ldif:
>>>> Command
>>>>>>>>> '/usr/bin/ldapmodify -v -f /tmp/tmplpfMNG -H
>>>>>>>>> ldap://freeipa02.mtl.ddistrict.int:389 -x -D cn=Directory Manager
>>>> -y
>>>>>>>>> /tmp/tmp4Svn9k' returned non-zero exit status 20
>>>>>>>>>  [26/31]: initializing group membership
>>>>>>>>>  [27/31]: adding master entry
>>>>>>>>>  [28/31]: configuring Posix uid/gid generation
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> the rest seems to work fine.
>>>>>>>>
>>>>>>>> You need to check ipareplica-install.log to see the real error.
>>>>>>>>
>>>>>>>> I wonder if "cn=ipa-http-delegation,cn=s4u2proxy,cn=etc,YOUR-SUFFIX"
>>>> and
>>>>>>>> "cn=ipa-ldap-delegation,cn=s4u2proxy,cn=etc,YOUR-SUFFIX" exist.
>>>>>>>>
>>>>>>>> Martin
>>>>>>>>
>>>>>>>
>>>>>>> The first one is there:
>>>>>>>
>>>>>>> ldapsearch -D "cn=Directory Manager” -W -LLL -x -b
>>>>>>> cn=ipa-http-delegation,cn=s4u2proxy,cn=etc,dc=ddistrict,dc=int""
>>>>>>> dn: cn=ipa-http-delegation,cn=s4u2proxy,cn=etc,dc=ddistrict,dc=int
>>>>>>> ipaAllowedTarget:
>>>>>> cn=ipa-ldap-delegation-targets,cn=s4u2proxy,cn=etc,dc=ddistr
>>>>>>>   ict,dc=int
>>>>>>> ipaAllowedTarget:
>>>>>> cn=ipa-cifs-delegation-targets,cn=s4u2proxy,cn=etc,dc=ddistr
>>>>>>>   ict,dc=int
>>>>>>> memberPrincipal: HTTP/freeipa01.prs.ddistrict.int at DDISTRICT.INT
>>>>>>> <mailto:HTTP/freeipa01.prs.ddistrict.int at DDISTRICT.INT>
>>>>>>> memberPrincipal: HTTP/freeipa02.prs.ddistrict.int at DDISTRICT.INT
>>>>>>> <mailto:HTTP/freeipa02.prs.ddistrict.int at DDISTRICT.INT>
>>>>>>> memberPrincipal: HTTP/freeipa02.mtl.ddistrict.int at DDISTRICT.INT
>>>>>>> <mailto:HTTP/freeipa02.mtl.ddistrict.int at DDISTRICT.INT>
>>>>>>> memberPrincipal: HTTP/freeipa01.chr.ddistrict.int at DDISTRICT.INT
>>>>>>> <mailto:HTTP/freeipa01.chr.ddistrict.int at DDISTRICT.INT>
>>>>>>> memberPrincipal: HTTP/freeipa01.bxl.ddistrict.int at DDISTRICT.INT
>>>>>>> <mailto:HTTP/freeipa01.bxl.ddistrict.int at DDISTRICT.INT>
>>>>>>> memberPrincipal: HTTP/freeipa01.mtl.ddistrict.int at DDISTRICT.INT
>>>>>>> <mailto:HTTP/freeipa01.mtl.ddistrict.int at DDISTRICT.INT>
>>>>>>> cn: ipa-http-delegation
>>>>>>> objectClass: ipaKrb5DelegationACL
>>>>>>> objectClass: groupOfPrincipals
>>>>>>> objectClass: top
>>>>>>>
>>>>>>>
>>>>>>> But not the second one:
>>>>>>>
>>>>>>> ldapsearch -D "cn=Directory Manager” -W -LLL -x -b
>>>>>>> cn=ipa-ldap-delegation,cn=s4u2proxy,cn=etc,dc=ddistrict,dc=int""
>>>>>>> No such object (32)
>>>>>>> Matched DN: cn=s4u2proxy,cn=etc,dc=ddistrict,dc=int
>>>>>>>
>>>>>>>
>>>>>>> Also what is strange is that I got the error only on one of the
>>>>>> replicas, the
>>>>>>> other one went through without any hiccups.
>>>>>>
>>>>>> Ok, I think I misguided you with the second DN, the real DN should be
>>>>>>
>>>> "cn=ipa-ldap-delegation-targets,cn=s4u2proxy,cn=etc,dc=ddistrict,dc=int",
>>>>>> see
>>>>>> /usr/share/ipa/replica-s4u2proxy.ldif for the LDIF that is being
>>>> loaded.
>>>>>>
>>>>>> The key here is to check the error message of ldapmodify that was run
>>>> on
>>>>>> the
>>>>>> failing replica, try to search in /var/log/ipareplica-install.log.
>>>>>>
>>>>>> Martin
>>>>>>
>>>>>
>>>>> Hi Martin,
>>>>>
>>>>> Finally got back on this problem.
>>>>>
>>>>> I seem to have a huge mess in my replication agreements between my
>>>> servers.
>>>>> if I run the "ipa-replica-manage list-ruv on my master which is
>>>>> freeipa01.prs,
>>>>>
>>>>> I get this:
>>>>> [root at freeipa01 ~]# ipa-replica-manage list-ruv
>>>>> freeipa01.prs.ddistrict.int:389: 4
>>>>> freeipa01.mtl.ddistrict.int:389: 16
>>>>> freeipa01.mtl.ddistrict.int:389: 13
>>>>> freeipa01.mtl.ddistrict.int:389: 12
>>>>> freeipa01.bxl.ddistrict.int:389: 10
>>>>> freeipa01.chr.ddistrict.int:389: 8
>>>>> freeipa01.mtl.ddistrict.int:389: 6
>>>>> freeipa02.prs.ddistrict.int:389: 3
>>>>> freeipa01.chr.ddistrict.int:389: 9
>>>>> freeipa02.mtl.ddistrict.int:389: 17
>>>>> freeipa02.mtl.ddistrict.int:389: 7
>>>>> freeipa02.mtl.ddistrict.int:389: 11
>>>>> freeipa02.mtl.ddistrict.int:389: 14
>>>>> freeipa02.mtl.ddistrict.int:389: 15
>>>>> [root at freeipa01 ~]#
>>>>>
>>>>>
>>>>> I've tried to do the ipa-replica-manage clean-ruv on all ID's relating
>>>> to
>>>>> freeipa02.mtl which is the one I'm having the most problems with and
>>>> would
>>>>> like to start from scratch.
>>>>>
>>>>> running the ipa-replica-manage list-clean-ruv gives me this:
>>>>>
>>>>> [root at freeipa01 slapd-DDISTRICT-INT]# ipa-replica-manage list-clean-ruv
>>>>> CLEANALLRUV tasks
>>>>> RID 11: Not all replicas online, retrying in 160 seconds...
>>>>> RID 17: Not all replicas online, retrying in 640 seconds...
>>>>> RID 7: Waiting to process all the updates from the deleted replica...
>>>>>
>>>>> No abort CLEANALLRUV tasks running
>>>>> [root at freeipa01 slapd-DDISTRICT-INT]#
>>>>>
>>>>> I'm kinda stuck in a loop and not sure which way to go.
>>>>
>>>> Check "ipa-replica-manage list" - some of the replicas listed here are not
>>>> active. You may have uninstalled a replica which is still pointed in this
>>>> list.
>>>>
>>>> I think /var/log/dirsrv/slapd-YOUR-REALM/errors contain additional
>>>> information
>>>> which replica is really not accessible.
>>>>
>>>>>
>>>>> I'm also stuck with a orphaned user in the WebUI which I see but can not
>>>>> delete, giving me the user doesn't exist.
>>>>>
>>>>> If I do an ldapsearch it seems incomplete:
>>>>> [root at freeipa01 ~]# ldapsearch -xLLL -D "cn=directory manager" -w
>>>> XXXXXXX
>>>>> -b dc=ddistrict,dc=int | grep -i arobitaille
>>>>> dn: cn=arobitaille,cn=groups,cn=compat,dc=ddistrict,dc=int
>>>>> cn: arobitaille
>>>>> memberUid: arobitaille
>>>>> dn: uid=arobitaille,cn=users,cn=compat,dc=ddistrict,dc=int
>>>>> homeDirectory: /home/arobitaille
>>>>> uid: arobitaille
>>>>> member:
>>>>> nsuniqueid=08165a01-dd3311e3-8982f534-a4dfcf64+uid=arobitaille,cn=user
>>>>> member:
>>>>> nsuniqueid=08165a01-dd3311e3-8982f534-a4dfcf64+uid=arobitaille,cn=user
>>>>> dn:
>>>>>
>>>> nsuniqueid=08165a01-dd3311e3-8982f534-a4dfcf64+uid=arobitaille,cn=users,cn
>>>>> homeDirectory: /home/arobitaille
>>>>> mepManagedEntry:
>>>> cn=arobitaille,cn=groups,cn=accounts,dc=ddistrict,dc=int
>>>>> mail: arobitaille at digital-district.ca
>>>>> krbPrincipalName: arobitaille at DDISTRICT.INT
>>>>> uid: arobitaille
>>>>> dn:
>>>>>
>>>> cn=arobitaille+nsuniqueid=08165a02-dd3311e3-8982f534-a4dfcf64,cn=groups,cn
>>>>> cn: arobitaille
>>>>> description: User private group for arobitaille
>>>>> mepManagedBy: uid=arobitaille,cn=users,cn=accounts,dc=ddistrict,dc=int
>>>>
>>>> This is a Directory Server replication conflict entry (notice the
>>>> nsuniqueid=08165a02-dd3311e3-8982f534-a4dfcf64 part), FreeIPA cannot
>>>> manipulate
>>>> those. You can try deleting this record with ldapdelete utility or any
>>>> LDAP gui
>>>> of choice.
>>>>
>>>> Martin
>>>>
>>>
>>> Hi Martin,
>>>
>>> I finally after a couple of hours managed to re-instate replication
>>> through all my replica. It's all working fine.
>>>
>>> Thanks for the insights.
>>>
>>> I just have one little orphaned user which has only the private group left
>>> behind.
>>>
>>> I'm not sure, since I'm still a newbie with ldapmodify/ldapdelete, how to
>>> get rid of those 2 entries:
>>>
>>> [root at freeipa01 ~]# ldapsearch -xLLL -D "cn=directory manager" -W -b
>>> cn=jdubreux,cn=groups,cn=accounts,dc=ddistrict,dc=int
>>>
>>> dn: cn=jdubreux,cn=groups,cn=accounts,dc=ddistrict,dc=int
>>>
>>> ipaUniqueID: ac27027c-84da-11e3-a4c4-c21e595ecd39
>>>
>>> mepManagedBy: uid=jdubreux,cn=users,cn=accounts,dc=ddistrict,dc=int
>>>
>>> cn: jdubreux
>>>
>>> objectClass: posixgroup
>>>
>>> objectClass: ipaobject
>>>
>>> objectClass: mepManagedEntry
>>>
>>> objectClass: top
>>>
>>> gidNumber: 871000045
>>>
>>> description: User private group for jdubreux
>>>
>>>
>>> [root at freeipa01 ~]# ldapsearch -xLLL -D "cn=directory manager" -W -b
>>> cn=jdubreux,cn=groups,cn=compat,dc=ddistrict,dc=int
>>>
>>> dn: cn=jdubreux,cn=groups,cn=compat,dc=ddistrict,dc=int
>>>
>>> objectClass: posixGroup
>>>
>>> objectClass: top
>>>
>>> gidNumber: 871000045
>>>
>>> cn: jdubreux
>>>
>>>
>>> After this I'm fully back on my feet!
>>>
>>>
>>> --
>>>
>>>
>>> Davis Goodman
>>> Directeur Informatique  |  IT Manager
>>>  [image: Digital-District] <http://www.digital-district.ca/>
>>>
>> I believe I  have found the syntax for removing the leftover private group
>> but I have an error thrown at me:
>>
>> [root at freeipa01 ~]# ldapmodify  -Y GSSAPI<<EOF
>>
>>
>> dn: cn=jdubreux,cn=groups,cn=accounts,dc=ddistrict,dc=int
>>
>> changetype:modify
>>
>> delete: objectclass
>>
>> objectclass: mepManagedEntry
>>
>>
>> delete:mepManagedBy
>>
>> EOF
>>
>> SASL/GSSAPI authentication started
>>
>> SASL username: admin at DDISTRICT.INT
>>
>> SASL SSF: 56
>>
>> SASL data security layer installed.
>>
>> modifying entry "cn=jdubreux,cn=groups,cn=accounts,dc=ddistrict,dc=int"
>>
>> *ldap_modify: Object class violation (65)*
>>
>> * additional info: attribute "mepManagedBy" not allowed*
>> This rings a bell?
>>
>> Version 3.0.0 of FreeIPA
>>
>> certmonger-0.61-3.el6.x86_64
>>
>> 389-ds-base-libs-1.2.11.15-32.el6_5.x86_64
>>
> 
> Hi David,
> 
> I am not sure what you are trying to do, but I think you want to just simply
> delete the objects you do not like, i.e. "changetype: delete" instead of
> "changetype: modify".
> 
> You can find more information for example here:
> http://www.zytrax.com/books/ldap/ch14/#ldapdelete
> 
> If you are not sure about the command syntax, try using some LDAP GUI. I for
> example use Apache Directory Studio. I also used Luma in the past, it is more
> lightweight.
> 
> HTH,
> Martin

Let me just close this thread, me and Davis had a short private conversation.
Davis was able to solve his issues by detaching the group from the user by
deleting both mepManagedEntry objectclass AND mepManagedBy attribute and then
deleting the group.

Martin

More information about the Freeipa-users mailing list