[Freeipa-users] DNS forwarding configuration randomly breaks and stops working

[nathan at nathanpeters.com]

>>> Looking at the log entries, it appears that there may have been a
>>> network
>>> connectivity 'blip' (maybe a switch or router was restarted) at some
>>> point
>>> and even after connectivity was restored, the global forwarding was
>>> failing because the "we can't contact our forwarder" status seemed to
>>> get
>>> stuck in memory.
>
> Most likely.
>
>>> [root at dc1 ~]# ipa dnsconfig-show
>>>   Global forwarders: 10.21.0.14
>>>   Allow PTR sync: TRUE
>
> This means that you are using the default forward policy which is 'first'.
> I.e. BIND daemon on the IPA server is trying to use the forwarder first
> and
> when it fails it fallbacks to asking server on the public Internet.
>
> I speculate that public servers know nothing about the name you were
> asking
> for and this negative answer got cached. This is default behavior in BIND
> and
> IPA did not change it.
>
> Workaround for network problems could be
> $ ipa dnsconfig-mod --forward-policy=only
> which will prevent BIND from falling back to public servers.
>
> Anyway, you should solve network connectivity problems, too :-)
>
> I hope this helps.
>
> --
> Petr^2 Spacek
>
> --
> Manage your subscription for the Freeipa-users mailing list:
> https://www.redhat.com/mailman/listinfo/freeipa-users
> Go to http://freeipa.org for more info on the project
>

Ok, we managed to figure out what was happening here, but I still think
there is a bug somewhere in the FreeIPA DNS components that is
exacerbating the issue.

We have split DNS in our company.  We have a public copy of our DNS
records, which contain only A records.  We also have an internal copy of
our DNS records, which contains a bunch of CNAME records.

When we use nslookup to query the IPA server for stash.externaldomain.net
NSLOOKUP returns that stash.externaldomain.net is a CNAME and it returns
both the CNAME and the associated A address.

When we query FreeIPA though a DNS client, FreeIPA returns that stash is a
CNAME and does not return the associated A address.  It seems like at that
point, FreeIPA decides that instead of sticking in 'forward' mode and
forwarding the request for the CNAME record to the forwarding server, it
decides instead to switch into recursive mode and begin the lookup by
querying root servers for externaldomain.net at which point since it is
going out to the general internet, it gets back answers from our public
facing DNS servers (remember we use split DNS).

The answer it gets is (quite rightly) that there is no such CNAME record
on the public DNS server.

So I have a couple questions about how the forward first policy is
supposed to work...

1.  In forward first mode, if the forward server returns a CNAME, is
FreeIPA supposed to ask the same forwarding server for the A record
associated with the CNAME, or is it supposed to then flip into recursive
mode and go to the root servers to find the CNAME?  I would expect #1, but
it seems like #2 is always happening no matter what.  I would only expect
it to attempt recursion if the initial query actually failed, not just
returned something other than an A record.

2. We did some more network packet capture, and noticed that in forward
first mode, the FreeIPA server, always sent out both a forward request to
the forwarding server, and an additional simultaneous request to the root
name servers (recursive mode).  It got back responses to both the
forwarded and recursive queries it had performed.  The recursive query
failed due to split DNS and the forwarded query succeeded due to it going
to an internal server which had the correct records.  Strangely enough...
the IPA server ignored the successful forwarded answer, and sent back the
'failed' answer it had gotten through recursion back to the requesting
client.  What is the behavior supposed to be in this situation and why is
the server always sending out the recursive request, even when it gets a
valid answer from the forwarded request?