[Freeipa-devel] Topology: Central node removal in star topology

[Petr Spacek]

On 24.6.2015 13:09, Ludwig Krispenz wrote:
> 
> On 06/24/2015 12:50 PM, Oleg Fayans wrote:
>>
>>
>> On 06/24/2015 12:28 PM, Ludwig Krispenz wrote:
>>>
>>> On 06/24/2015 12:02 PM, Oleg Fayans wrote:
>>>>
>>>>
>>>> On 06/24/2015 11:47 AM, Ludwig Krispenz wrote:
>>>>>
>>>>> On 06/24/2015 11:36 AM, Oleg Fayans wrote:
>>>>>>
>>>>>>
>>>>>> On 06/24/2015 11:25 AM, Ludwig Krispenz wrote:
>>>>>>> Oleg,
>>>>>>>
>>>>>>> the topology plugin relies on existing connection between servers which
>>>>>>> remain in a topolgy. If you remove a central node in your topology you
>>>>>>> are asking for trouble.
>>>>>>> With Petr's patch it warns you that your topology will be disconnected,
>>>>>>> and if you insist we cannot guarantee anything.
>>>>>> Agree. I just wanted to try edge cases to see how one can break the
>>>>>> system :)
>>>>>>> should we completely prohibit this ? I don't know, I think you could
>>>>>>> also enforce an uninstall of vm175 with probably the same result.
>>>>>>> what you mean be calculating the remaining topology and send it to the
>>>>>>> remaining servers does not work, it would require to send a removal of
>>>>>>> a segment, which would be rejected.
>>>>>>>
>>>>>>> The topology is broken, and I don't know how much we should invest in
>>>>>>> making this info consistent on all servers.
>>>>>>>
>>>>>>> More interesting would be if we can heal this later by adding new
>>>>>>> segments.
>>>>>> Yes, here comes the biggest question raised from this case: obviously,
>>>>>> when none of the nodes possess the correct topology information
>>>>>> (including the one which deleted the central node), there is no way to
>>>>>> fix it by adding segments connecting the nodes that became disconnected. 
>>>>> It shoul not need the full information, but it has to be able to reach
>>>>> one of the nodes to be connected. when the topology is broken, you loose
>>>>> to feature to be ably to apply a change on any node, eg in your case if
>>>>> you want to connect vm036 and vm056 an have removed vm175, you have to do
>>>>> it on vm056, vm036 or vm244. This should work, if not we have to fix it -
>>>>> unless we completely prevent disconnecting a topology
>>>> Well, this is exactly the problem here: all replicas should contain
>>>> precise copies of all the info: accounts, hosts, sudorules, etc, including
>>>> topology information. However, if in this case I manually connect
>>>> disconnected node at vm127 (or vm056, does not matter) it results in
>>>> topology information inconsistency across the infrastructure:
>>>> This would be the topology from the point of view of vm127:
>>> did you add teh connection on vm127 or on vm244 ? sorry, but in these
>>> situations to understand what's going on, it can matter.
>>> to me it looks like you did it on vm127, so its there, it got replicated to
>>> vm244, but replicationback does not work and so the deletion of teh segs to
>>> vm175, which should still be in the changelogs of 036 and 244, don#t get to
>>> 127. Do you have something in the error logs of 244 ?
>> Yes, I added the connection on vm127. vm244 does not have anything in the
>> ldap errors log corresponding to the replication with vm127. In fact, I
>> tried to create a user on vm244 to see if it will be replicated to vm127,
>> and the user creation failed with the following error message:
>> Operations error: Allocation of a new value for range cn=posix
>> ids,cn=distributed numeric assignment plugin,cn=plugins,cn=config failed!
>> Unable to proceed.
>>
>> Is it because the master node was deleted?
> think so, yes.
> There are probably more things to check before removing a server :-(

This particular error is caused by the way how we distribute DNA ranges among
servers. The range is assigned only on first use (not during replica
installation) so when the original master is gone you have no way how to
obtain the range (if you did not need it before).

This is tracked as
https://bugzilla.redhat.com/show_bug.cgi?id=1211366

Please comment here so we do not forget how annoying it is :-)

Petr^2 Spacek

>> The corresponding message in the error log is
>> [24/Jun/2015:12:44:18 +0200] dna-plugin - dna_pre_op: no more values
>> available!!