[Freeipa-devel] DNS zone serial number updates [#2554]

[Petr Spacek]

Hello,

I found following reply in unsent mail, so I resending it to complete original 
thread.

It's just "for record", because original idea was obsoleted by recent 
discussion on Thursday meeting.

Petr^2 Spacek

On 04/18/2012 05:53 PM, Simo Sorce wrote:
> On Wed, 2012-04-18 at 17:21 +0200, Petr Spacek wrote:
>
>>> If this happens, it is possible that on one of the masters the serial
>>> will be updated twice even though no other change was performed on the
>>> entry-set. That is not a big deal though, at most it will cause a
>>> useless zone transfer, but zone transfer should already be somewhat rate
>>> limited anyway, because our zones do change frequently due to DNS
>>> updates from clients.
>> SOA record has also refresh, retry and expiry fields. These define how often
>> zone transfer should happen. It's nicely described in [8].
>
> Sure but we have 2 opposing requirements. On one hand we want to avoid
> doing too many transfers too often. On the other we want to reflect
> dynamic Updates fast enough to avoid stale entries in the slaves. So the
> problem is that we need to carefully balance and tradeoff.
>
>> There is next problem with transfers: Currently we support only full zone
>> transfers (AXFR), not incremental updates (IXFR), because there is no "last
>> record change"<->SOA# information. For now it's postponed, because nobody
>> wanted it.
>
> We can support IXFR, using entryUSN, all entries that have a entryUSN
> higher than the max we recorded at last trabfer time would need to be
> transfered. It's not urgent, but we have a way to do it relatively
> easily. I think the main issue here would be deleted records. We might
> need to access the tombstone for it, or keep some record of deleted
> entries somewhere, needs more investigation.
Ticket for this enhancement created:
https://fedorahosted.org/bind-dyndb-ldap/ticket/64

It definitely needs investigation, but I think we can postpone it for now.

>>> You still need to search the whole cache and save additional data. (I
>>> sure hope you do not keep in memory the whole ldap object but a parsed
>>> version of it, if you keep the whole LDAP object I think we just found
>>> another place for enhancement. Wasting all that memory is not a good
>>> idea IMO).
>> Only DNS records are stored, i.e. parsed objects.
>
> Excellent.
>
>> Please, can you explain "You still need to search the whole cache and save
>> additional data."? I probably missed some important point.
>
> I was referring to your proposal to store the modifyTimestamp in the
> cache. But I think we agree modifyTimestamp is not the way to go so we
> can ignore the rest.
Ah, it is misunderstanding. I meant "store only max(modifyTimestamp)", not 
each separate modifyTimestamp value.
It doesn't matter, modifyTimestamp is not the right way.

>>>> There are still problems to solve without DS plugin (specifically
>>>> mapping/updating NN part from YYYYMMDDNN), but: Sounds this reasonable?
>>>
>>> Well I am not sure we need to use a YYYYMMDDNN convention to start with.
>>> I expect with DYNDNS updates that a 2 digit NN will never be enough,
>>> plus it is never updated by a human so we do not need to keep it
>>> readable. But I do not care eiither way, as long as the serial can
>>> handle thousands of updates per day I am fine (if this is an issue we
>>> need to understand how to update the serial in time intervals).
>> Current BIND implementation handles overflow in one day gracefully:
>> 2012041899 ->  2012041900
>> So SOA# can be in far future, if you changes zone too often :-)
>
> Sure, but then what's the point of keeping it in date format ? :-)
I agree, it doesn't make sense. I consulted this with Adam on IRC: We don't 
have to follow this syntax.

>> AFAIK this format is traditional, but not required by standard, if arithmetic
>> works. [9] defines arithmetic for SOA serials, so DS plugin should follow it.
>>
>> It says "The maximum defined increment is 2147483647 (2^31 - 1)"
>> This limit applies inside to one SOA TTL time window (so it shouldn't be a
>> problem, I think). I didn't looked into in this RFC deeply. Some practical
>> recommendations can be found in [10].
>
> Yeah 2^31 is large enough for practical deployments if you start small,
> if you start close to the top (2012 is not the far from 2147) then you
> have substantially reduced the window.
I didn't described it well.

(2^31 - 1) is maximal *increment* inside one SOA expiry time window, not 
maximal value. Time window is typically between days and weeks.

SOA serial number range is 0..(2^32 - 1). Defined arithmetic allows to 
overflow (2^32 - 1) -> 0 gracefully (with proper result = make a transfer to 
slaves).

Only limitation is (2^31 - 1) increment during SOA expiry time window.

For 4 weeks SOA expire timer:
2^31/(60*60*24*7*4) =  887 updates per second during 4 weeks

So there is no real limitation, we can forget to this problem I think.


I created bind-dyndb-ldap ticket for SOA update inside DS:
https://fedorahosted.org/bind-dyndb-ldap/ticket/65

Petr^2 Spacek

>
>> Thanks for your time.
>
> YW
>
> Simo.
>