[Freeipa-devel] DNS zone serial number updates [#2554]

[Simo Sorce]

On Wed, 2012-04-18 at 17:21 +0200, Petr Spacek wrote:

> > If this happens, it is possible that on one of the masters the serial
> > will be updated twice even though no other change was performed on the
> > entry-set. That is not a big deal though, at most it will cause a
> > useless zone transfer, but zone transfer should already be somewhat rate
> > limited anyway, because our zones do change frequently due to DNS
> > updates from clients.
> SOA record has also refresh, retry and expiry fields. These define how often 
> zone transfer should happen. It's nicely described in [8].

Sure but we have 2 opposing requirements. On one hand we want to avoid
doing too many transfers too often. On the other we want to reflect
dynamic Updates fast enough to avoid stale entries in the slaves. So the
problem is that we need to carefully balance and tradeoff.

> There is next problem with transfers: Currently we support only full zone 
> transfers (AXFR), not incremental updates (IXFR), because there is no "last 
> record change"<->SOA# information. For now it's postponed, because nobody 
> wanted it.

We can support IXFR, using entryUSN, all entries that have a entryUSN
higher than the max we recorded at last trabfer time would need to be
transfered. It's not urgent, but we have a way to do it relatively
easily. I think the main issue here would be deleted records. We might
need to access the tombstone for it, or keep some record of deleted
entries somewhere, needs more investigation.

> > You still need to search the whole cache and save additional data. (I
> > sure hope you do not keep in memory the whole ldap object but a parsed
> > version of it, if you keep the whole LDAP object I think we just found
> > another place for enhancement. Wasting all that memory is not a good
> > idea IMO).
> Only DNS records are stored, i.e. parsed objects.

Excellent.

> Please, can you explain "You still need to search the whole cache and save 
> additional data."? I probably missed some important point.

I was referring to your proposal to store the modifyTimestamp in the
cache. But I think we agree modifyTimestamp is not the way to go so we
can ignore the rest.

> > This is why using the modifyTimestamp is not workable in this case.
> Ok, I didn't realized these problems. Now I know why DNS has single master :-D

Heh, hard problems need courage, it is easy to hide under the
single-master rock :-)

> > no the problem is not backdating the SOA serial, the problem is *not*
> > updating it when new entris become available because they were "in the
> > past". So if no other changes are made to DNS a zone transfer may not
> > kick at all indefinitely even though the master has new/changed entries.
> > This would cause a long term de-synchronization of the slaves I think is
> > not really acceptable.
> I agree with your long-term de-synchronization point, but with dynamic updates 
> is not really probable.

Keep in mind that dynamic updates can be disabled, we cannot count on
them. If we have them it will "help" (but also cause more load), but we
need to work well enough when they are completely absent.

> > I was already expecting that, but you cannot distinguish modifyTimestamp
> > per attribute, only per object, so if modifyTimestamp is changed for an
> > attribute you do not care about you still have to count it.
> AFAIK you can watch changes only for selected attributes (through psearch).

True, but if the connection drops, then you need a new full search and
there you'll pick up all the new timestamps. So you can have
inconsistent behavior depending on DS restarts, named restarts, and so
on. Not a nice corner to paint ourselves in.

> >> There are still problems to solve without DS plugin (specifically
> >> mapping/updating NN part from YYYYMMDDNN), but: Sounds this reasonable?
> >
> > Well I am not sure we need to use a YYYYMMDDNN convention to start with.
> > I expect with DYNDNS updates that a 2 digit NN will never be enough,
> > plus it is never updated by a human so we do not need to keep it
> > readable. But I do not care eiither way, as long as the serial can
> > handle thousands of updates per day I am fine (if this is an issue we
> > need to understand how to update the serial in time intervals).
> Current BIND implementation handles overflow in one day gracefully:
> 2012041899 -> 2012041900
> So SOA# can be in far future, if you changes zone too often :-)

Sure, but then what's the point of keeping it in date format ? :-)

> AFAIK this format is traditional, but not required by standard, if arithmetic 
> works. [9] defines arithmetic for SOA serials, so DS plugin should follow it.
> 
> It says "The maximum defined increment is 2147483647 (2^31 - 1)"
> This limit applies inside to one SOA TTL time window (so it shouldn't be a 
> problem, I think). I didn't looked into in this RFC deeply. Some practical 
> recommendations can be found in [10].

Yeah 2^31 is large enough for practical deployments if you start small,
if you start close to the top (2012 is not the far from 2147) then you
have substantially reduced the window.

> Thanks for your time.

YW

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York