[Freeipa-devel] DNS zone serial number updates [#2554]

[Petr Spacek]

Hello,

first of all - snippet moved from the end:
 > I think we need to try to be more consistent than what we are now. There
 > may always be minor races, but the current races are too big to pass on
 > IMHO.
 >
I definitely agree. Current state = completely broken zone transfer.

Rest is in-line.

On 04/17/2012 06:13 PM, Simo Sorce wrote:
> On Tue, 2012-04-17 at 17:49 +0200, Petr Spacek wrote:
>> Hello,
>>
>> there is IPA ticket #2554 "DNS zone serial number is not updated" [1],
>> which is required by RFE "Support zone transfers in bind-dyndb-ldap" [2].
>>
>> I think we need to discuss next steps with this issue:
>>
>> Basic support for zone transfers is already done in bind-dyndb-ldap. We
>> need second part - correct behaviour during SOA serial number update.
>>
>> Bind-dyndb-ldap plugin handles dynamic update in correct way (each
>> update increment serial #), so biggest problem lays in IPA for now.
>>
>> Modifying SOA serial number can be pretty hard, because of DS
>> replication. There are potential race conditions, if records are
>> modified/added/deleted on two or more places, replication takes some
>> time (because of network connection latency/problem) and zone transfer
>> is started in meanwhile.
>>
>> Question is: How consistent we want to be?
>
> Enough, what we want to do is stop updating the SOA from bind-dyndb-ldap
> and instead update it in a DS plugin. That's because a DS plugin is the
> only thing that can see entries coming in from multiple servers.
> If you update the SOA from bind-dyndb-ldap you can potentially set it
> back in time because last write win in DS.
>
> This will require a persistent sarch so bind-dyndb-ldap can be updated
> with the last SOA serial number, or bind-dyndb-ldap must not cache it
> and always try to fetch it from ldap.

Bind-dyndb-ldap has users with OpenLDAP. I googled a bit and OpenLDAP should 
support Netscape SLAPI [3][4], but I don't know how hard is to code 
interoperable plugin.
Accidentally I found existing SLAPI plugin for "concurrent" BIND-LDAP backend 
project [5].


Can we think for a while about another ways? I would like to find some (even 
sub-optimal) solution without DS plugin, if it's possible and comparable hard 
to code.

>>   Can we accept these
>> absolutely improbable race conditions? It will be probably corrected by
>> next SOA update = by (any) next record change. It won't affect normal
>> operations, only zone transfers.
>
> Yes and No, the problem is that if 2 servers update the SOA
> independently you may have the serial go backwards on replication. See
> above.
>
>> (IMHO we should consider DNS "nature": In general is not strictly
>> consistent, because of massive caching at every level.)
>
> True, but the serial is normally considered monotonically increasing.
I agree. How DS will handle collisions? When same attribute is modified 
independently at two places? It's simply overwritten by one of values? I can't 
find information about this at directory.fedoraproject.org.

(Side question: It's a real big problem? If it's result of very improbable 
race condition? It will broke zone transfer, but next zone update will correct 
this. As result of this failure last change is not transferred to slave DNS 
servers, before another zone update takes place.)

>> If it's acceptable, we can suppress explicit SOA serial number value in
>> LDAP and derive actual value from latest modifyTimestamp value from all
>> objects in cn=dns subtree. This approach saves some hooks in IPA's LDAP
>> update code and will save problems with manual modifications.
>
> It will cause a big search though. It also will not take in account when
If we use persistent search it's not a problem. Persistent search dumps whole 
DB to RBT in BIND's memory and only changes are transferred after this initial 
query.
We can compute maximal modifyTimestamp from all idnsRecord objects during 
initial query. Any change later in time will add only single attribute to 
transfer + max(currvalue, newvalue) operation. This way (with psearch) has 
reasonably small overhead, I think.

> there are changes replicated from another replica that are "backdated"
> relative to the last modifyTimestamp.
If we maintain max(modifyTimestamp) value whole time, new backdated values 
will not backdate SOA, because max(modifyTimestamp) can't move back.

It's not correct behaviour also, I know. But again: It's result of improbable 
race condition and next zone update will correct this situation.

> Also using modifyTimestamp would needlessly increment the SOA if there
> are changes to the entries that are not relevant to DNS (like admins
> changing ACIs, or other actions like that).
I think it's not a problem. Only consequence is unnecessary zone transfer. How 
often admin changes ACI?

If we want to save this overhead, we can count max(modifyTimestamp) only for 
idnsRecord objects (and only for known attributes) - but I think it's not 
necessary.


There are still problems to solve without DS plugin (specifically 
mapping/updating NN part from YYYYMMDDNN), but: Sounds this reasonable?

Petr^2 Spacek

>> Persistent search will be (probably) required for effective implementation.
>> I think it's not a problem, because DNSSEC will require (with very high
>> probability) persistent search for generating NSEC/NSEC3 records.
>>
>> [1] https://fedorahosted.org/freeipa/ticket/2554
>>
>> [2] https://bugzilla.redhat.com/show_bug.cgi?id=766233
>>
>>
>> Please, post your opinions about DNS consistency strictness.
>
> I think we need to try to be more consistent than what we are now. There
> may always be minor races, but the current races are too big to pass on
> IMHO.
>
> Simo.
>

[3] 
http://opensource.apple.com/source/OpenLDAP/OpenLDAP-143/OpenLDAP/include/slapi-plugin.h

[4] http://www.openldap.org/lists/openldap-devel/200111/msg00124.html

[5] http://thewalter.net/stef/software/slapi-dnsnotify/