[Freeipa-devel] DNA range distribution to replicas by default

[Petr Spacek]

On 11.6.2015 16:11, Simo Sorce wrote:
> On Thu, 2015-06-11 at 12:38 +0200, Petr Spacek wrote:
>> On 9.6.2015 15:06, Simo Sorce wrote:
>>> On Tue, 2015-06-09 at 10:30 +0200, Petr Spacek wrote:
>>>> Hello,
>>>>
>>>> I would like to discuss
>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1211366
>>>> "Error creating a user when jumping from an original server to replica".
>>>>
>>>> Currently the DNA ranges are distributed from master to other replicas on
>>>> first attempt to get a number from particular range.
>>>>
>>>> This works well as long as the original master is reachable but fails
>>>> miserably when the master is not reachable for any reason.
>>>>
>>>> It is apparently confusing to users [1][2] because it is counter-intuitive.
>>>> They have created a replica to be sure that everything will work when the
>>>> first server is down, right?
>>>>
>>>> Remediation is technically simple [3] (just assign a range to the new replica)
>>>> but it is confusing to the users, error-prone, and personally I feel that this
>>>> is an unnecessary obstacle.
>>>>
>>>> It seems to me that the original motivation for this behavior was that the
>>>> masters were not able to request range back from other replicas when a local
>>>> range was depleted.
>>>>
>>>> This deficiency is tracked as
>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1029640 and it is slated for fix
>>>> in 4.2.x time frame.
>>>>
>>>> Can we distribute ranges to the replicas during ipa-replica-install when we
>>>> fix bug 1029640?
>>>
>>> That was not the only reason, another reason is that you do not want to
>>> distribute and fragment ranges to replicas that will never be used to
>>> create users. What we should do perhaps, is to automatically give a
>>> range to CA enabled masters so that at least those servers have a range.
>>> If all your CAs are unavailable you have major issues anyway.
>>>
>>> Though it is a bit bad to have magic behaviors, maybe we should have a
>>> "main DNA range holder" role that can be assigned to arbitrary servers
>>> (maybe the first replica gets it by default), and when done the server
>>> acquire part of the range if it has none.
>>
>> This concept sounds good to me!
>>
>> I would only reverse the default, i.e. distribute ranges by default to all
>> replicas and let admin to toggle a knob if he feels that his case really needs
>> to limit range distribution.
> 
> By the time you *feel* that it may be too late.
> 
>>> Another option is that a replica can instantiate a whole new range if
>>> all the range bearing servers are not around, but that also comes with
>>> its own issues.
>>>
>>> In general I wouldn't want to split by default, because in domains with
>>> *many* replicas most of them are used for load balancing and will never
>>> be used to create users, so the range would be wasted.
>>
>> This should not be an issue when
>> https://bugzilla.redhat.com/show_bug.cgi?id=1029640 is fixed because replicas
>> will be able to request range back if the local chunk is depleted.
>>
>> Is that correct?
> 
> To some degree, the main issue is when replicas get removed abruptly and
> are not around to "give back" anything.
> We would need to start working on a range-scavenging tool to reclaim
> "lost" ranges if you go and automatically distribute ranges to every
> replica that ever pops up.

Okay, I understand that.

I can't help myself but it seems to me that this problem is inherent to
current design and can always happen because the range information is local to
the replica. As a result, if the replica with a range disappears we always
need to do some sort of manual recovery to get the free numbers back.

Consequently, lowering number of replicas with ranges just makes the problem
less common but does not eliminate it.

Let's look at: cn=posix-ids,cn=dna,cn=ipa,cn=etc,dc=ipa,dc=example

It seems that we already have information which replicas have free values in
the shared tree - this is good, but not sufficient to eliminate the problem.
The information about range start/end and the next free value is missing in
the shared tree and is stored only in cn=config on particular replica.

It seems to me that adding this range start/end values to the shared tree
would help because the information about the range would be preserved even if
the replica was deleted/lost.

Apparently the attribute dnaRemainingValues in the shared tree is updated
after each number allocation so adding the next free value (to a new
attribute) to the shared tree would not add any significant replication churn
because the object needs to be updated anyway.

What did I miss?

-- 
Petr^2 Spacek