[Freeipa-devel] [PATCH 0064] Create ipa-otp-decrement 389DS plugin

[Nathaniel McCallum]

On Wed, 2014-10-08 at 21:45 +0200, thierry bordaz wrote:
> On 10/08/2014 07:30 PM, Nathaniel McCallum wrote:
> > On Wed, 2014-10-08 at 17:30 +0200, thierry bordaz wrote:
> >> On 10/07/2014 06:00 PM, Nathaniel McCallum wrote:
> >>> Attached is the latest patch. I believe this includes all of our
> >>> discussions up until this point. However, a few bits of additional
> >>> information are needed.
> >>>
> >>> First, I have renamed the plugin to ipa-otp-counter. I believe all
> >>> replay prevention work can land inside this plugin, so the name is
> >>> appropriate.
> >>>
> >>> Second, I uncovered a bug in 389 which prevents me from validating the
> >>> non-replication request in bepre. This is the reason for the additional
> >>> betxnpre callback. If the upstream 389 bug is fixed, we can merge this
> >>> check back into bepre. https://fedorahosted.org/389/ticket/47919
> >>>
> >>> Third, I believe we are now handling replication correct. An error is
> >>> never returned. When a replication would cause the counter to decrease,
> >>> we remove all counter/watermark related mods from the operation. This
> >>> will allow the replication to apply without decrementing the value.
> >>> There is also a new bepost method which check to see if the replication
> >>> was discarded (via CSN) while having a higher counter value. If so, we
> >>> apply the higher counter value.
> >> For me the code is good. It took me some time to understand the benefit
> >> of removing mods in preop.
> >> In fact I think it is a good idea, as it prevents extra repair ops and
> >> also make more easy the computation of the value to set in repair mod.
> >>> Here is the scenario. Server X receives two quick authentications;
> >>> replications A and B are sent to server Y. Before server Y can process
> >>> server X's replications, an authentication is performed on server Y;
> >>> replication C is sent to server X. The following logic holds true:
> >>>    * csnA < csnB < csnC
> >>>    * valueA = valueC, valueB > valueC
> >>>
> >>> When server X receives replication C, ipa-otp-counter will strip out all
> >>> counter mod operations; applying the update but not the lower value. The
> >>> value of replication B is retained. This is the correct behavior.
> >>>
> >>> When server Y processes replications A and B, ipa-otp-counter will
> >>> detect that a higher value has a lower CSN and will manually set the
> >>> higher value (in bepost). This initiates replication D, which is sent to
> >>> server X. Here is the logic:
> >>>    * csnA < csnB < csnC < csnD
> >>>    * valueA = valueC, valueB = valueD, valueD > valueC
> >>>
> >>> Server X receives replication D. D has the highest CSN. It has the same
> >>> value as replication B (server X's current value). Because the values
> >>> are the same, ipa-otp-counter will strip all counter mod operations.
> >>> This reduces counter write contention which might become a problem in
> >>> N-way replication when N>2.
> >> I think we should rather let the mods going on. So  that the full
> >> topology will have
> >> valueD (or valueB)/csnD rather having a set of servers having
> >> valueD/csnB and an other set valueD/csnD.
> > I think you misunderstand. The value for csnD is only discarded when the
> > server already has valueB (valueB == valueD). Only the value is
> > discarded, so csnD is still applied. The full topology will have either
> > valueB w/ csnD or valueD w/ csnD. Since, valueB always equals valueD, by
> > substitution, all servers have valueD w/ csnD.
> >
> > Nathaniel
> >
> 
> There are several parts where the CSN are stored.
> One is used to allow replication protocol to send the approriate 
> updates. This part is stored into a dedicated entry: RUV.
> In fact when the update valudD/CSND will be received and applied, the 
> RUV will be updated with csnD.
> 
> An other part is the attribute/attribute values. An attribute value 
> contains the actual value and the CSN associated to that value.
> This CSN is updated by entry_apply_mod_wsi when it decides which value 
> to keep and which CSN is associated to this value.
> 
> In the example above, on the server X, the counter attribute has 
> valueB/csnB. Then it receives the update ValueD/csnD it discard this 
> update because valueD=ValueB. That means that on serverX we will have 
> valueB/csnB.

It does not discard the update (CSN). It discards the value because
valueD == valueB. So csnD will be applied, it just won't touch the
counter values; valueB will be retained.

> Now if on an other server we receive the updates in the reverse order: 
> valueD/csnD first then valueB/csnB.
> This server will apply and valueD/csnD then will discard valueB/csnB.

This server will apply valueD/csnD AND csnB, but not valueB. This is
because valueB == valueD.

> ValueD and ValueB being identical it is not a big issue. But we will 
> have some server with csnD and others with csnB.

As I understand my code, all servers will have csnD. Some servers will
have valueB and others will have valueD, but valueB == valueD.

We *never* discard a CSN. We only discard the counter/watermark mods in
the replication operation.

Nathaniel