[Freeipa-devel] topology issues
thierry bordaz
tbordaz at redhat.com
Wed Jun 10 12:13:13 UTC 2015
On 06/10/2015 10:51 AM, Ludwig Krispenz wrote:
>
> On 06/10/2015 10:41 AM, Martin Basti wrote:
>> On 10/06/15 09:13, Ludwig Krispenz wrote:
>>> Hi,
>>>
>>> there seems to be somethin going wrong in the code to delete the
>>> services.
>>>
>>> The code is:
>>>
>>> # delete master entry with all active services
>>> try:
>>> dn = DN(('cn', replica), ('cn', 'masters'), ('cn', 'ipa'),
>>> ('cn', 'etc'), self.suffix)
>>> entries = self.conn.get_entries(dn, ldap.SCOPE_SUBTREE)
>>> if entries:
>>> entries.sort(key=lambda x: len(x.dn), reverse=True)
>>> for entry in entries:
>>> self.conn.delete_entry(entry)
>>> except errors.NotFound:
>>> pass
>>> except Exception, e:
>>> if not force:
>>> raise e
>>> elif not err:
>>> err = e
>>>
>>> In the access log we see:
>>>
>>>
>>> [09/Jun/2015:08:32:42 -0400] conn=150 op=52 SRCH
>>> base="cn=f22replica1.bagam.net,cn=masters,cn=ipa,cn=etc,dc=bagam,dc=net"
>>> scope=2 filter="(objectClass=*)" attrs=ALL
>>> [09/Jun/2015:08:32:42 -0400] conn=150 op=52 RESULT err=0 tag=101
>>> nentries=8 etime=0 notes=U
>>>
>>> this was the get_entries, it returns 8 entries
>>>
>>> [09/Jun/2015:08:32:42 -0400] conn=150 op=53 DEL
>>> dn="cn=KDC,cn=f22replica1.bagam.net,cn=masters,cn=ipa,cn=etc,dc=bagam,dc=net"
>>> [09/Jun/2015:08:32:42 -0400] conn=150 op=53 RESULT err=0 tag=107
>>> nentries=0 etime=0 csn=5576dceb000600040000
>>> [09/Jun/2015:08:32:42 -0400] conn=150 op=54 DEL
>>> dn="cn=KPASSWD,cn=f22replica1.bagam.net,cn=masters,cn=ipa,cn=etc,dc=bagam,dc=net"
>>> [09/Jun/2015:08:32:42 -0400] conn=150 op=54 RESULT err=0 tag=107
>>> nentries=0 etime=0 csn=5576dceb000700040000
>>> [09/Jun/2015:08:32:42 -0400] conn=150 op=55 DEL
>>> dn="cn=MEMCACHE,cn=f22replica1.bagam.net,cn=masters,cn=ipa,cn=etc,dc=bagam,dc=net"
>>> [09/Jun/2015:08:32:43 -0400] conn=150 op=55 RESULT err=0 tag=107
>>> nentries=0 etime=1 csn=5576dcec000100040000
>>>
>>> here it stops after deleting three entries, and it should do it in
>>> reverse order of the dn length, but KDC is deleted before MEMCACHE
Something surprising is that according to the code
# delete master entry with all active services
try:
dn = DN(('cn', replica), ('cn', 'masters'), ('cn', 'ipa'),
('cn', 'etc'), self.suffix)
entries = self.conn.get_entries(dn, ldap.SCOPE_SUBTREE)
if entries:
entries.sort(key=lambda x: len(x.dn), reverse=True)
for entry in entries:
self.conn.delete_entry(entry)
except errors.NotFound:
pass
except Exception, e:
if not force:
raise e
elif not err:
err = e
try:
entry = *self.conn.get_entry*(
DN(('cn', 'ipa'), ('cn', 'etc'), self.suffix), ['aci'])
sub = {'suffix': self.suffix, 'fqdn': replica}
...
We should see a search on cn=ipa,cn=etc,SUFFIX following the deletion of
those entries.
But the next op is an UNBIND.
Is that the code executed by ipa-replica-manage ?
>>> [09/Jun/2015:08:32:43 -0400] conn=150 op=56 UNBIND
>>>
>>>
>>> Are there any ideas what is going on or how to debug it ?
>>>
>> Actually, the both DNs of KDC and MEMCACHE has the same length.
>> IPA implements own DN class, where length is the number of AVA/RDN
>> parts (mixed in code, but it means the 'cn=user' has length 1, and
>> 'cn=user,cn=accounts' has length 2)
>>
>> def __len__(self):
>> return len(self.rdns)
>>
>> This reverse sort guarantees the child entries will be removed before
>> the parent entries.
> thanks, then it is ok, but it does not explain why not all services
> and the master were not deleted.
>>
>> To debug, maybe print the entries from IPA code, before sort and
>> after sort might help.
> yep, but so far only Oleg reprted this, and he's not here today, I
> haven't reproduced the issue
>>
>> Martin^2
>>
>>>
>>> On 06/09/2015 05:32 PM, Ludwig Krispenz wrote:
>>>> Hi Oleg,
>>>> thanks for access to your machine, the replication agreements are
>>>> still there - and that is expected since the server was not removed.
>>>>
>>>> In the access log I see:
>>>>
>>>> [09/Jun/2015:08:32:42 -0400] conn=150 op=52 SRCH
>>>> base="cn=f22replica1.bagam.net,cn=masters,cn=ipa,cn=etc,dc=bagam,dc=net"
>>>> scope=2 filter="(objectClass=*)" attrs=ALL
>>>> [09/Jun/2015:08:32:42 -0400] conn=150 op=52 RESULT err=0 tag=101
>>>> nentries=8 etime=0 notes=U
>>>> [09/Jun/2015:08:32:42 -0400] conn=150 op=53 DEL
>>>> dn="cn=KDC,cn=f22replica1.bagam.net,cn=masters,cn=ipa,cn=etc,dc=bagam,dc=net"
>>>> [09/Jun/2015:08:32:42 -0400] conn=150 op=53 RESULT err=0 tag=107
>>>> nentries=0 etime=0 csn=5576dceb000600040000
>>>> [09/Jun/2015:08:32:42 -0400] conn=150 op=54 DEL
>>>> dn="cn=KPASSWD,cn=f22replica1.bagam.net,cn=masters,cn=ipa,cn=etc,dc=bagam,dc=net"
>>>> [09/Jun/2015:08:32:42 -0400] conn=150 op=54 RESULT err=0 tag=107
>>>> nentries=0 etime=0 csn=5576dceb000700040000
>>>> [09/Jun/2015:08:32:42 -0400] conn=150 op=55 DEL
>>>> dn="cn=MEMCACHE,cn=f22replica1.bagam.net,cn=masters,cn=ipa,cn=etc,dc=bagam,dc=net"
>>>> [09/Jun/2015:08:32:43 -0400] conn=150 op=55 RESULT err=0 tag=107
>>>> nentries=0 etime=1 csn=5576dcec000100040000
>>>> [09/Jun/2015:08:32:43 -0400] conn=150 op=56 UNBIND
>>>>
>>>> the search for cn=f22replica1.bagam.net,cn=masters,.... returns 8
>>>> entries, which then should be deleted, but only 3 ae deleted and the
>>>> cn=f22replica1.bagam.net,cn=masters,... entry is not deleted, so
>>>> the topology segments are not deleted, and the agreement is not
>>>> removed.
>>>>
>>>> I don't know why ipa-replica-manage del does stop deleting services
>>>> and the master entry
>>>>
>>>>
>>>>
>>>> On 06/09/2015 04:25 PM, Oleg Fayans wrote:
>>>>>
>>>>>
>>>>> On 06/09/2015 04:19 PM, Ludwig Krispenz wrote:
>>>>>>
>>>>>> On 06/09/2015 04:14 PM, Oleg Fayans wrote:
>>>>>>>
>>>>>>>
>>>>>>> On 06/09/2015 04:04 PM, Ludwig Krispenz wrote:
>>>>>>>>
>>>>>>>> On 06/09/2015 03:55 PM, Oleg Fayans wrote:
>>>>>>>>> Hi everybody,
>>>>>>>>>
>>>>>>>>> The current status of Topology plugin testing is as follows:
>>>>>>>>>
>>>>>>>>> 1. There is still no proper way of removing the replica.
>>>>>>>>> Standard procedure using `ipa-replica-manage del` throws
>>>>>>>>> "Server is unwilling to perform: Entry is managed by topology
>>>>>>>>> plugin.Deletion not allowed.".
>>>>>>>> yes, that is for the first attempt to directly remove the
>>>>>>>> agreement, but when the server is removed the agreements should
>>>>>>>> be removed
>>>>>>> We should probably think of less threatening error message in
>>>>>>> this case. Just from reading the command output one might
>>>>>>> conclude that replica removal failed.
>>>>>>>>> The replication agreement though does get deleted,
>>>>>>>> then it is ok,
>>>>>>>>> but the topology information does not get updated.
>>>>>>>> what do you mean, where do you check ? in the "remaining"
>>>>>>>> topology the shared tree should be updated, for the removed
>>>>>>>> replica it will not, but this should be uninstalled anyway
>>>>>>> The problem here, is that the topology information does not get
>>>>>>> updated on master as well.
>>>>>> could you be a bit more precise. what do you still see ? the
>>>>>> agreement will be only removed if the segment is removed, and
>>>>>> this should be reoplicated to all severs in the remaining
>>>>>> topology - if you don't disconnect it by removing the replica.
>>>>>> and what was the topology structure and which replica did you
>>>>>> remove, on which server did you remove it?
>>>>> So, Here is the results of the `topologysegment-find` command
>>>>> before replica removal:
>>>>> root at f22master:/var/log/dirsrv/slapd-BAGAM-NET]$ ipa
>>>>> topologysegment-find
>>>>> Suffix name: realm
>>>>> ------------------
>>>>> 2 segments matched
>>>>> ------------------
>>>>> Segment name: f22master.bagam.net-to-f22replica1.bagam.net
>>>>> Left node: f22master.bagam.net
>>>>> Right node: f22replica1.bagam.net
>>>>> Connectivity: both
>>>>>
>>>>> Segment name: f22master.bagam.net-to-f22replica2.bagam.net
>>>>> Left node: f22master.bagam.net
>>>>> Right node: f22replica2.bagam.net
>>>>> Connectivity: both
>>>>> ----------------------------
>>>>> Number of entries returned 2
>>>>> ----------------------------
>>>>> Then, after issuing `ipa-replica-manage-del f2replica1.bagam.net
>>>>> --force` on the master, the same command on master still shows
>>>>> exactly the same topology:
>>>>>
>>>>> root at f22master:/var/log/dirsrv/slapd-BAGAM-NET]$ ipa
>>>>> topologysegment-find
>>>>> Suffix name: realm
>>>>> ------------------
>>>>> 2 segments matched
>>>>> ------------------
>>>>> Segment name: f22master.bagam.net-to-f22replica1.bagam.net
>>>>> Left node: f22master.bagam.net
>>>>> Right node: f22replica1.bagam.net
>>>>> Connectivity: both
>>>>>
>>>>> Segment name: f22master.bagam.net-to-f22replica2.bagam.net
>>>>> Left node: f22master.bagam.net
>>>>> Right node: f22replica2.bagam.net
>>>>> Connectivity: both
>>>>> ----------------------------
>>>>> Number of entries returned 2
>>>>> ----------------------------
>>>>>
>>>>>>>>> When I then issue `ipa topologysegment-del`, it fails due to
>>>>>>>>> "ipa: ERROR: Server is unwilling to perform: Removal of
>>>>>>>>> Segment disconnects topology.Deletion not allowed."
>>>>>>>> correct, you can only do it after removal of the server
>>>>>>> I do not get it. Master still thinks it has the replica, it
>>>>>>> displays it both in CLI using `ipa topologysegment-find` and in
>>>>>>> the web-ui. (although it does not show it using `ipa host-find`,
>>>>>>> which is correct), and there is no way to manually make it
>>>>>>> change it's mind?
>>>>>>>>>
>>>>>>>>> I tried to disable the segment first and then delete it, but
>>>>>>>>> with the segment properly disabled, the attempt to delete it
>>>>>>>>> raised a GSS error: "ipa: ERROR: Kerberos error: Kerberos
>>>>>>>>> error: ('Unspecified GSS failure. Minor code may provide more
>>>>>>>>> information', 851968)/('KDC returned error string:
>>>>>>>>> PROCESS_TGS', -1765328324)/". I am not sure, where to search
>>>>>>>>> for corresponding logs. The session transcript is attached.
>>>>>>>>>
>>>>>>>>> 2. The following is probably unrelated to the topology plugin:
>>>>>>>>> I installed a replica with --setup-ca option. Then, on this
>>>>>>>>> replica tried to prepare another replica:
>>>>>>>>> -------------------------------------------------------------------------------------------------------------------------------------------------
>>>>>>>>>
>>>>>>>>> root at f22replica2:/home/ofayans/f22]$ ipa-replica-prepare
>>>>>>>>> --ip-address 192.168.122.141 f22replica3.bagam.net
>>>>>>>>> Directory Manager (existing master) password:
>>>>>>>>>
>>>>>>>>> Preparing replica for f22replica3.bagam.net from
>>>>>>>>> f22replica2.bagam.net
>>>>>>>>> Creating SSL certificate for the Directory Server
>>>>>>>>> Certificate issuance failed
>>>>>>>>> -------------------------------------------------------------------------------------------------------------------------------------------------
>>>>>>>>>
>>>>>>>>> The corresponding line in the dirsrv log:
>>>>>>>>> [09/Jun/2015:09:54:46 -0400] - Entry
>>>>>>>>> "uid=admin,ou=people,o=ipaca" -- attribute "krbExtraData" not
>>>>>>>>> allowed
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Oleg Fayans
>>>>>>> Quality Engineer
>>>>>>> FreeIPA team
>>>>>>> RedHat.
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Oleg Fayans
>>>>> Quality Engineer
>>>>> FreeIPA team
>>>>> RedHat.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>> --
>> Martin Basti
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20150610/d51d7283/attachment.htm>
More information about the Freeipa-devel
mailing list