[Freeipa-devel] full BIND view support [was: routing requests to local servers - DNS SRV]

[Petr Spacek]

Hello,

I found RFE for this feature:
https://bugzilla.redhat.com/show_bug.cgi?id=815621
It was filled against IPA component in BZ so I didn't find it up to now.

On 05/29/2012 05:36 PM, Simo Sorce wrote:
> On Tue, 2012-05-29 at 17:16 +0200, Petr Spacek wrote:
>> Discussion about major changes should be read as "design for far future".
>>
>> On 05/25/2012 04:10 PM, Simo Sorce wrote:
>>> On Thu, 2012-05-24 at 19:07 +0200, Petr Spacek wrote:
...snip ...
>>>> This request actually means "differentiate answer to DNS query on client's IP
>>>> address basics".
>>>> Relevant thread on ipa-users-list:
>>>> https://www.redhat.com/archives/freeipa-users/2012-April/msg00070.html
>>>>
>>>> First question is: Do we want to implement it?
>>>> (IMHO it is very important for large-scale deployments.)
>>>
>>> I am not really sure I like it, as it makes things quite difficult to
>>> handle.
>>>
>>> Please think how we are going to operate if you ahve a view dfined and a
>>> client does a dyn DNS update.
...snip ...

Merged with proposal from another thread:

On 05/29/2012 05:58 PM, William Brown wrote:
> I think Simo is on the money here. Adding this to bind-dyndb-ldap makes
> the most sense. Views become a multivalued attribute, and configurable
> via an idnsView object of some kind (It may even be some kind of "group"
> membership even?). This would only need to be in the idnsZone
> information - All records in a zone are bounded by the policies of that
> zone. There is no concept of a per-record view. It would also means that
> records can be correctly updated for DynDNS updates correctly.
>
> If memory serves correctly, Named impelements views in such a way that
> you update the record of the view you are updating with dyndns. So, lets
> say you have a view public and private. Lets also say you were sharing a
> zone between these azone.example (You were using the view to apply
> attributes like recursive query permissions). When the DynDNS update
> comes in, from a host matched in the "private" view, the code will go to
> update the zone that the the private view references. In this case, the
> update would be referenceable by both views.
>
> In the case where you have azone.example in public and private, but each
> has their own unique zone file, the dyndns update would only update the
> zone that the private zone has access to - The public view is still
> referencing the other data.
If I understood correctly, you propose to share all records or nothing. If I 
didn't miss something, it equals to current behaviour. You can define two 
views with same base DN or different DN - and share all or nothing.

Main problem is how to create and maintain "slightly" different zones, e.g. 
SRV records with different priorities. With plain BIND you can create two zone 
files and use separate file and $INCLUDE for shared records.

I will explore BIND's behaviour with two zones containing shared part.

Proposed approaches should allow sharing at record level in our LDAP scheme 
(see my reply below). Or - missed I something in you text?


> This functionality could easily be achieved by the plugin. When the Dns
> update comes in, and that data is referenced by 2 views you have to
> choose - Do you duplicate the record, with each record now representing
> the data in each view, or do you update the record that is accesible by
> both views? From the FreeIPA standpoint of consistency and simplicity, I
> think that updating the record, provided the update comes from a view
> that is able to make updates, would be a sensible solution. Spliting the
> record is much hassle for no real benefit. This keeps the DNS data
> intact, and consistent across all views. Views would really only affect
> access policies to zones, recursive responders etc.
>
> The best benefit of this, would be that policies of "views" could be
> edited with the CLI tool or the web interface, rather than having to
> edit the named.conf file. This would again simplify administration of
> DNS services.
> -- Sincerely, William Brown
Yes, it is a big benefit of LDAP configuration.

>> ...snip ...
>>
>>>> Adam and I discussed possible approaches. We agreed on "stackable" approach:
>>>> - Store whole original DNS tree in one subtree, let say "base".
>>>> - Create "override" subtrees for each "locality" = set of customized records.
>>>> Shared records are only in "base". During DNS query processing "override"
>>>> subtree is searched first. If record does not exist in "override" subtree,
>>>> search will continue in "base" subtree.
>>>
>>> Yes, this is my first thought too.
>>>
>>>> Second question is how to realize these "overrides":
...snip ...

>>> The simple way is to do subtree searches, if you get back more than one
>>> result for a specific name then you know you have views and proceed to
>>> filter out the one you want. The bonus here is that you can cache all
>>> replies if you keep different caches per view.
>>>
>>> The alternative is to add a 'viewname' or something in the filter, but
>>> then you need to do 2 searches and fallbacks. This sounds more
>>> expensive.
>>>
>>>> It do not require any change in bind-dyndb-ldap code. All merges/overrides
>>>> will be done on Directory server.
>>>
>>> Given we do persistent searches and we also do some caching in
>>> bind-dyndb-ldap we almost certainly do not want to 'fool' it by
>>> returning different values from DS w/o bind-dyndb-ldap knowledge.
>> I probably missed something, I don't see the problem. One bind-dyndb-ldap
>> instance is run for each "view" separately. Each instance has own caches and
>> other data structures. They don't know about each other.
>
> This is a problem, 389DS uses a lot fo resources for persistent
> searches. It means if you have views we'd have to disable persistent
> searches or enable them only in some views.
I don't understand why 389DS requires a lot of resources in this case. My 
(naive) imagination says: "Do normal search during initial query and then send 
any updates to client as soon as they comes in." What I missed? (I really 
don't know about 389 internals, so sorry for naive questions.)

> Why do you need to run multiple instances ? Do each of them have its own
> in memory cache ? How much memory are we going to sue with many views ?
Each "instance" is completely independent. I can't explain this decision in 
it's full deep, this design was done a long time ago (before I started).

> I think we really need to discuss also these architectural issues.
I checked the code: Full view support will require bigger changes, but it is 
doable. (All necessary BIND-plugin APIs are in place.)

Re-implementing all configuration options from named.conf can be a bit difficult.
http://www.zytrax.com/books/dns/ch7/view.html contains ~ 60 different view 
options, ~ 33 of them are not present in zones.

General question is: What we want to do with named.conf? Is our target to 
re-implement all named.conf options in plugin/LDAP? Configuration is quite 
complex and some general rule how to do it is necessary for interface consistency.

... snip ...
>>>> - Another approach is to add support directly to bind-dyndb-ldap, but it is
>>>> not so nice solution.
>>>
>>> Why ?
>>>
>>>>    In that case both LDAP search bases have to be defined
>>>> in /etc/named.conf. For each DNS query is necessary to search "override" base
>>>> first. If required record is not present then is necessary to search through
>>>> "base" subtree.
>>>
>>> No, you do not need to do multiple searches, you just need to filter the
>>> results by view.
>> You are right, if we change the way how DNS data are stored in LDAP (a bit).
>> Mentioned idea was to create new subtree ("cn=dns-viewname") and re-use
>> existing driver and all tools for managing it with minimal changes.
>
> I see too many pitfalls, seem like not a good tradeoff in this case.
No way is really simple and straight-forward. I'm not against plugin side 
implementation generally, it allows better configuration management and 
simpler record-sharing.

>>> It would make it very easy to manage if we added a 'dnsView' attribute
>>> to overriding entries in the subtree. This would allow us to define a
>>> new overriding entry and even assign it to multiple views if the
>>> 'dnsView' attribute is multivalued.


>> Merging with another thread:
>> On 05/25/2012 09:20 PM, Simo Sorce wrote:
>>> On Fri, 2012-05-25 at 15:52 +0200, Petr Spacek wrote:
>>>>>   On 05/24/2012 08:00 PM, Dmitri Pal wrote:
... snip ...

>>>>>   Current DNS "name" (name can potentially contain multiple records)
>>>>>   structure
>>>>>   is following:
>>>>>
>>>>>   dn: idnsname=_kerberos._tcp,idnsname=e.localnet,cn=dns,dc=e,dc=org
>>>>>   objectClass: idnsrecord
>>>>>   objectClass: top
>>>>>   idnsName: _kerberos._tcp
>>>>>   sRVRecord: 0 100 88 unused-4-107
>>>>>
>>>>>   DNS name is part of DN. It is not possible to have more objects with
>>>>>   same DNS
>>>>>   name and different attributes. This problem lead me to "stackable"
>>>>>   approach.
>>> Yes, and we can also use multiple attributes in the same tree, although
>>> for clarity I probably prefer the subtree approach.
>>>
>>> So a few options:
>>>
>>> 1. all in the same subtree:
>>> # Normal object
>>> dn: idnsname=bar,idnsname=foo.org,cn=dns,dc=foo,dc=org
>>> objectClass: idnsrecord
>>> objectClass: top
>>> idnsName: bar
>>> aRecord: 192.168.12.34
>>> dNSTTL: 1200
>>>
>>> # Object belongin to the 'DMZ' view
>>> dn:cn=DMZ-bar,idnsname=foo.org,cn=dns,dc=foo,dc=org
>>> objectClass: idnsrecord
>>> objectClass: top
>>> objectClass: nsContainer
>>> cn: DMZ-bar
>>> idnsName: bar
>>> aRecord: 5.6.7.8
>>> dNSTTL: 3600
>>> idnsView: DMZ
>>>
>>>
>>> NOTE: I had to add nsContainer here in order to give the object a way to
>>> have a unique name by using the CN attribute. I am not very fond of this
>>> arrangement though. It is also ugly to parse out using a LDAP browser.
>>> It make one thing simpler in that using multiple values for dnsView you
>>> can assign the same entry to multiple views.
Advantage can be speed: You can do subtree search with base 
idnsname=foo.org,cn=dns,dc=foo,dc=org, so only necessary part of whole LDAP 
tree is walked through. Filter can contain only idnsView condition without a 
idnsName.
(Current code does LDAP search with base idnsname=foo.org,cn=dns,dc=foo,dc=org 
and scope "base". Is it good approach?)

We can create idnsRecordOverride class with single attribute idnsViewGroupID 
or similar to avoid necessity of nsContainer.

It allows to use "(|(idnsView=view1)(!(idnsView=*)))" filter also. It can 
replace extendedMatch filter mentioned below, it is not necessary to walk 
through objects not related to "target" view or default.


>>> 2. using per view subtrees
>> Generally - I like this idea.
>>
>>> # Normal object
>>> dn: idnsname=bar,idnsname=foo.org,cn=dns,dc=foo,dc=org
>>> objectClass: idnsrecord
>>> objectClass: top
>>> idnsName: bar
>>> aRecord: 192.168.12.34
>>> dNSTTL: 1200
>> I prefer to create "_default" view for normal records (as BIND does).
>> e.g. dn: idnsname=bar,cn=_default,idnsname=foo.org,cn=dns,dc=foo,dc=org
>>
>> It allows to treat "normal" and override records in the same way. Also it
>> allows to optimize query with extensibleMatch filter.
>> E.g. plugin is configured to serve records for "view1": Filter can be
>> (|(cn:dn:=view1)(cn:dn:=_default)) so LDAP server will not return unnecessary
>> records for view2, view3, view4 ...
>
> I don;t think that filter is standard LDAP, I do not want to rely on
> something like that.
http://tools.ietf.org/html/rfc4511#section-4.5.1.7.7

> It is preferrable to have a idnsView attribute in the entry, so you can
> use a normal filter and index on it.
>
>> Another problem to solve is how to modify SOA record/hide zone/add new zone to
>> view. With this view-zone-dnssubtree ordering you cannot modify SOA record or
>> hide the zone from certain views.
>
> Please provide more info, it is not clear to me why you can't do it.
I probably misunderstood original text. I can nest idnsZone object to cn=view 
subtree ...

Most basic question is how to merge "view" data with "base" data: What about 
simplest approach - make override at "name" (= LDAP object) level?
E.g. "base" object:
dn: idnsname=bar,idnsname=foo.org,cn=dns,dc=foo,dc=org
objectClass: idnsrecord
objectClass: top
idnsName: bar
aRecord: 192.168.12.34
tXTRecord: "blah blah"
dNSTTL: 1200

"view" object:
dn: idnsname=bar,cn=view,idnsname=foo.org,cn=dns,dc=foo,dc=org
objectClass: idnsrecord
objectClass: top
idnsName: bar
aRecord: 11.22.33.44

Query for bar.foo.org will be resolved (through view) to:
bar	<default TTL>	IN	A	11.22.33.44
(There is default TTL and no TXT record.)

If I think about override at record (= LDAP attribute) level, I don't see a 
way how to delete a record.

>> With ordering zone-view-dnssubtree situation is better. We still can run
>> subtree query in 'cn=dns' and as bonus it is possible to define zone only in
>> some views or create modified version of zone's SOA record in another views.
>>
>> Problem is how to *not* override zone SOA record. One (not very nice) approach:
>> We can create extensibleObject and define container only with  idnsName
>
> extensibleObject is not ok to use except for experimentation.
>
>> attribute (necessary for RDN construction). "Real" zones from "fake" ones can
>> be distinguished by objectClass. Records are leaves under this zone record as
>> usual.
>
> Create a idnsViewZone objectclass or something like that.
It is better approach, definitely.

>>> # Object belongin to the 'DMZ' view
>>> dn:idnsname=bar,cn=DMZ,idnsname=foo.org,cn=dns,dc=foo,dc=org
>>> objectClass: idnsrecord
>>> objectClass: top
>>> idnsName: bar
>>> aRecord: 5.6.7.8
>>> dNSTTL: 3600
>>>
>>>
>>> NOTE: I prefer this method as it makes things a lot easier to manage and
>>> view through an LDAP broiwser, however it makes sharing entries between
>>> multiple views a bit awkward.
>> I like this idea. What about LDAP aliases? (I never used them, so I don't
>> know.) It is not very nice, but for limited amount of "override" records it
>> can be sufficient.
>
> Obviously this feature will be abused, so if you start with assuming
> "limited" you are already on the wrong path IMO :-)
Ok, now we are talking about full view support. Not the original (limited) 
"routing requests to local servers".


>>> 3. using only one 'views' subtree pr zone and dnsView to discrimnate
>>>
>>> # Normal object
>>> dn: idnsname=bar,idnsname=foo.org,cn=dns,dc=foo,dc=org
>>> objectClass: idnsrecord
>>> objectClass: top
>>> idnsName: bar
>>> aRecord: 192.168.12.34
>>> dNSTTL: 1200
>>>
>>> # Object belongin to the 'DMZ' view
>>> dn:idnsUniqueID=F6A1245-bar,cn=views,idnsname=foo.org,cn=dns,dc=foo,dc=org
>>> objectClass: idnsrecord
>>> objectClass: top
>>> idnsUniqueID: F6A1245-bar
>>> idnsName: bar
>>> aRecord: 5.6.7.8
>>> dNSTTL: 3600
>>> idnsView: DMZ
>>> idnsView: VPN
>>>
>>>
>>> NOTE: here I added also a idnsUniqueID as a way to have unique names so
>>> we can have multiple entries for the same record. This is so that you
>>> can have 3 different entries for the same record belonging to 3
>>> different views. The reason why I added the actual name after a random
>>> id is that this way it is simpler to recognize what it is when looking
>>> at an ldap browser w/o having to read the actual object attributes, it
>>> also make collisions a lot less likely and so it allows to keep the
>>> random part smaller (and thus more readable).
>>> Also note that I've put 2 values in idnsView, meaning that this record
>>> belongs to 2 separate views. This allows compact representation when
>>> multiple views want to redefine some records in the same way (an dothers
>>> in a different way, thus why 2 separate views)/
>> I also prefer "way 2", as I said above.
> I actually prefer 3 :)
> Simo.

After some time I started to prefer variant 1. Only disadvantage is a bit 
worse reading (for humans :-) in LDAP browser. I tried it in my LDAP browser 
(Apache LDAP studio):

With search filter (|(cn=view1)(cn=view3)) and return attribute list 
"tXTRecord, cn" it draws nice table:

"DN"	"tXTRecord"	"cn"
"cn=view1,idnsname=test2,idnsname=ee.localnet,cn=dns,dc=e,dc=org"	"text for 
view 1+3"	"view1|view3"
"cn=view1,idnsname=test,idnsname=ee.localnet,cn=dns,dc=e,dc=org"	"text view 
1"	"view1"

Oh, text wrapping is not perfect for ASCII art ... See the attachment. With 
click on DN I can jump directly to found record.

Yes, you has to do search operation. I think you has to do search even with 
"way three" on real databases. Multi-valued idnsView attribute can make 
eye-browsing hard, because override record<->view pairing is not obvious from DN.

For debugging purposes you probably use search to limit whole tree to 
problematic parts only and so on ... (I have testing DB with 60k records and 
it is really not good idea to use LDAP browser without limits to walk through 
it ...)

For mentioned reasons I prefer "way 1".


Petr^2 Spacek
-------------- next part --------------
A non-text attachment was scrubbed...
Name: view_search.png
Type: image/png
Size: 16798 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20120531/b47d0e25/attachment.png>