[libvirt] Proposal: no dnsmasq (no dhcp and no dns) and no radvd option

[Gene Czarcinski]

On 11/25/2012 05:08 PM, Laine Stump wrote:
> On 11/25/2012 08:25 AM, Gene Czarcinski wrote:
>> On 11/24/2012 04:19 PM, Laine Stump wrote:
>>> On 11/23/2012 02:18 PM, Gene Czarcinski wrote:
>>>> On 11/20/2012 05:29 PM, Laine Stump wrote:
>>>>> On 11/20/2012 02:36 PM, Gene Czarcinski wrote:
>>>>>>> Laine mentioned something yesterday that got me to thinking: being
>>>>>>> able to specify that dnsmasq is not to be started for an interface.
>>>>>>>
>>>>>>> Let me expand that by saying that libvirt would not start dnsmasq
>>>>>>> for
>>>>>>> either dns or dhcp and also would not start radvd.  However, the
>>>>>>> IPv4
>>>>>>> and IPv6 gateway addresses would be defined on the virtual network
>>>>>>> interface and the "usual" iptables and ip6tables rules would be in
>>>>>> force.
>>>>>>> This would allow a user to configure dnsmasq to meet any user
>>>>>>> desires
>>>>>>> or use something completely different instead of dnsmasq.
>>>>>>>
>>>>>>> Questions:  Useful?  Worth the time and effort?
>>>>> That was already determined before I mentioned it to you - it's been
>>>>> requested several times, and I've told some people it was "going to
>>>>> happen", although didn't say when:-).
>>>>>
>>>> OK, color this "almost done" (doc and schemas need updating).
>>>>
>>>> Three new parameters are added:
>>>>         <network disableDnsmasq='yes'  logDnsQueries='yes'
>>>> logDhcp='yes' >
>>> Don't refer to dnsmasq specifically in the XML unless it's something
>>> that is unique to dnsmasq (and then we should seriously consider  if
>>> we're thinking about it the right way before adding it). The intent is
>>> that the XML should describe the desired configuration in a manner high
>>> level enough that a new driver using different backend components would
>>> be able to use the same XML attributes and arrive at similar
>>> functionality. For example, there is active work right now to support
>>> libvirt's server side on FreeBSD, and they may decide that they want to
>>> implement their networks using different implementations of DNS and dhcp
>>> servers (they will already need to do something different in place of
>>> Linux host brdges and iptables, so it will have to be a completely new
>>> driver anyway).
>>>
>>> Obviously there are cases where something specific to the backend needs
>>> to be in the XML; for example the virtio-net drivers can specify <driver
>>> name='qemu' ...> and in that case can have an option ioeventfd and
>>> event_idx options which would probably be meaningless to any netdev
>>> driver for another hypervisor, but we aim to keep that to a minimum, and
>>> use it only when the characteristic described really is unique to
>>> particular backend).
>>>
>>> (At any rate, it seems reasonable to me that someone may want us to
>>> provide a dhcp server but no dns server. Yes, this also implies that we
>>> should have a way in the config to feed in the address of an alternative
>>> DNS server ("--dhcp-option=option:dns-server,x.x.x.x" and
>>> "--dhcp-option=option6:dnsserver,xxxx::xxxx") to send to dhcp clients
>>> when they've disabled dnsmasq's own dns server (in case you hadn't found
>>> it in the manpage yet - you can run dnsmasq with no dns server by adding
>>> "--port=0" to the commandline). As soon as I hit send on this, I'm going
>>> to type up a short mail asking for comments on how to add support for
>>> any standard dhcp option.
>>>
>>> As for the other two - any reason why you used "logDnsQueries" instead
>>> of "logDns"? Also, do you think we might want the ability to specify a
>>> level in the future? (I guess if we did, that could coexist with
>>> yes|no).
>>>
>>> And while we're on the topic of logging: one thing that's bothered me a
>>> lot, and we've had other people complain about it as well, is that every
>>> single DHCP lease renewal results in a message to /var/log/messages, and
>>> there doesn't seem to be any way to suppress it. Is that correct?
>> OK, I had not realized that it was possible to not have dns while
>> continuing to do dhcp.  So, it now becomes disableDns="yes" which
>> results in --port=0 being passed to dnsmasq.  For completeness, I
>> suggest that disableDhcp="yes" and disableRA="yes" be added.
> disableDhcp already exists - you just don't have a <dhcp> section (and
> if you do have a <dhcp> section, it seems very illogical that you would
> want dhcp disabled).
Yes, you are correct.  I was adding that for the symmetry and 
completeness ;)
>
> disableRA may be a valid concern, since that's on by default if there
> are any IPv6 addresses.
>
>
> However, to pull the discussion back in - I had earlier suggested *not*
> having an attribute called "disableXXX" because it starts to get
> confusing - you're saying "Yes, I don't want this feature.". It's
> simpler to understand "*No*, I *don't* want this feature". And I'd also
> suggested making it an option of the already existing <dns> element (see
> below).
I understand your point and will use an "enable" user interface with the 
default being that, in general, things are enabled unless "enableXX='no' 
is specified.
>
>> I have changed my mind and will make it logDns=, logDhcp= and (again
>> for completeness) logRA= ... this last one may require a small dnsmasq
>> update.
> I like "<dns log='xxx'.../>" (edit: I may have changed my opinion on
> this - see the end of the message)
And where would you suggest logDhcp adding logDhcp?  Remember that 
things done with dnsmasq are for an interface and not an address. If I 
specify log='yes' for an IPv4 dhcp specification, then I will also get 
it for any IPv6 DHCP also ... and maybe that is not all bad.
>
>> However, I still like the idea of not starting anything and, as I
>> state below, you can start dnsmasq even if it is configured to do nothing.
> Why? That would only make sense if it was a placeholder that could be
> modified later to do something (and even then only if it would be
> impossible to just start it at a later time instead).
I was attempting to honor the point made by David Allen in that bugzilla 
report that more or less stated some users (certainly not most) would 
want complete control over dns/dhcp/ra services.
>
>>    Therefore, I propose adding "disableDaemons=" which is not specific
>> to any package but would mean not to start either dnsmasq or radvd
>> regardless of whether or not they would be started.
> I dislike that idea - it give two ways to specify the same thing, which
> is bad. Experience has shown that it's better to have a single way to
> specify each thing; that way there is no ambiguity when doing a
> parse->format->parse cycle.
>
> Remember that you can easily have no dnsmasq daemon by simply specifying:
>
>     <dns enable='no'>
>
> and then having no <dhcp> element anywhere.
That is not true.  Specifying --port=0 disables the dns and with no dhcp 
range or host specified there will be no dhcp but dnsmasq can be 
started.  If you do not want dns but do want dhcp then --port=0 is the 
right way to do it.  I can implement that if enableDns='no' and there is 
no dhcp specified, then dnsmasq will not be started.
>
> As for RA, that will require an option to turn off. We currently only
> start radvd if networkRadvdConfContents returns a non-NULL string, and
> it is *perfectly* setup to have an "enableRa='yes|no'" (with default of
> 'yes') on each IP address - it would simply not append the lines for an
> IP address to the conffile string if that IP's enableRa was no. So, if
> all the ipv6 addresses had enableRa='no', radvd wouldn't be run (and I'm
> sure you could do something similar for the ra implementation in dnsmasq).
I am not sure about dnsmasq being "tunable" with respect to RA.  I will 
have to check the code and ask Simon Kelley.

I understand that you can define multiple IPv4 and multiple IPv6 gateway 
addresses on a network interface but only one IPv4 DHCP and one IPv6 
DHCP.  I can see the need for both IPv4 and IPv6 protocols on a single 
network "fabric" but I am not sure how many real network "fabrics" have 
multiple subnetworks on them.  Yes, it could be done but I am not 
certain why you would do that (and I am also sure that someone has a 
very valid reason for doing that).
>
>
>> Lease renewal messages should not be that frequent it is probably a
>> "small matter of programming" to get an option to suppress them ... if
>> Simon Kelley would add that.
> An indicator that something more than just log='yes|no' may be desired.
>
>> Speaking of logging, if you do not like the lease renewal messages,
>> then you will have a bigger problem with the "RTR-ADVERT" messages
>> that dnsmasq issues every 3//4/5 seconds for every started interface.
> Yes, that's definitely a no-go. I'm pretty sure nobody would stand for
> that. If that's how dnsmasq operates, we can't switch to using dnsmasq
> until that's remedied.
I agree ... all those messages are a non starter!  I am hoping that 
there will be a log-ra parameter since that should be handled by the 
capability code you are adding.

But, I also want to get log-dhcp "fixed" because it mostly determines 
the formation and not "if" a syslog message is generated.  [See 
dnsmasq-discussion]

>
>>    I have submitted a "tiny" patch to dnsmasq which ties issuing this
>> message to log-dhcp being enabled.
>
> That will be good, but since it will be something not detectable from
> the --help output, it will mean that use of dnsmasq for RA will need to
> be dependent on an "even later" version of dnsmasq.
That is why I really want "log-ra" added so that it will appear in 
dnsmasq --help
>
>
>>>> If nothing is specified, then the default is no, no, no so that things
>>>> work as they do now.  Currently, if the boolean value is 'no' or
>>>> false, then that parameter is not written out.  I could easily change
>>>> that depending on what others want to do.
>>>>
>>>> Laine, I changed this to be disableDnsmasq because that is the real
>>>> result.  If this is specified, then starting radvd will also be
>>>> suppressed.  It is as if no IPv4 or IPv6 addresses were specified.
>>>> However, the iptables and ip6tables rules will remain the same since
>>>> it is assumed that gateway addresses will be specified.
>>> Actually that's yet another switch that we need to add - the ability to
>>> suppress all iptables rules for a particular network (unless it is in
>>> NAT mode, in which case it makes no sense, because it's the difference
>>> in rules that changes it from mode='route' to mode='nat'). Perhaps it
>>> could be something like filter='none|open|restricted', although I don't
>>> know if that's descriptive enough - even without getting into specific
>>> protocols (which I don't think should be done - at that point they
>>> should really be using a guest-specific filterref) you might still want
>>> to know if there should be any rules at all, allow everything, allow
>>> outgoing sessions but not incoming, allow inter-guest traffic but not
>>> incoming from the outside, allow nothing (all of these could be
>>> overridden by a particular guest's filterref)
>> Easy enough to implement but what is the purpose.  If there are no
>> rules at all (and perhaps no gateway addresses), then the interface
>> will not work for guest-to-guest communications either.
> Because some people want libvirt-created networks, but want to manage
> the firewall rules themselves.
OK.  Understood and I agree.  However, should this be specified in the 
<network> or <interface>?
>
> (btw, you must be running with net.bridge.bridge-nf-call-iptables=1,
> otherwise communications between guests (ipv4 and ipv6) would work just
> fine with no extra rules)
Did not know that (there are a lot of things I do not know or 
understand).  I will have to do some testing.

BTW, if this is true, why are the IPv4 (iptables) rules added when there 
are no IP addresses (v4 or v6) specified on an network interface?

I need to figure out what is setting this (or is it a default) because:

sysctl -a | grep "bridge-nf-call"

results in:
net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1

So yes, the rule is needed.

>
>> But wait, if there are no rules at all the the "default"
>> "--reject-with icmp-port-unreachable" will not be there either so
>> maybe things will work.  However, I thought that filters were
>> specified as part of the <interface> definition, not the <network>
>> definition.
> Yes, exactly. The network would have no rules of its own (or maybe a
> ruleset to allow only incoming, as is currently the case for a NATed
> network), and each guest's interface could override that with its own
> rules to allow incoming.
>
>> So, where should this be handled?  It makes some sense to me that for
>> the types of networks src/network/bridge_driver.c handles, maybe
>> <network> is more appropriate.
> More appropriate than what? Than using a <filterref> in each guest's
> interface? Or more appropriate than making it an option of <forward>? I
> think it makes sense to have this be an option of <forward>, because it
> is controlling exactly what traffic is allowed to be forwarded. And when
> you look at the existing forwarding modes:
>
> <forward mode='nat'>
>
> It doesn't make sense to have different levels of filtering for a NATed
> network, because the filter rules are required for NAT to work at all,
> and due to the way NAT works, incoming sessions are impractical anyway:
>
>     none - nonsensical (if you're not going to allow any traffic in-out,
>            then why have <forward> at all?)
>     open - nonsensical (incoming sessions can't work without additional
>            forwarding rules anyway)
>     restricted - that's what it does anyway
>
> <forward mode='bridge|hostdev'>
>
> These networks have nothing managed by libvirt anyway
>
> <forward mode='none'> (i.e. "no <forward> element)
>
> By definition if you have this type of network, you don't want to allow
> any traffic in or out of the network (heck, we even run dnsmasq with
> different options so that the dns server won't forward requests for
> unknown hosts).
>
> <forward mode='route'>
>
> This is the *only* type of forwarding where it makes sense to offer
> different options for the stock filters provided by libvirt.
>
>     none - the host admin will be managing filters themselves external to
> libvirt.
>
>     open - this is what it currently does - allows incoming and outgoing
> sessions
>
>     restricted - this would be similar to what is currently done in NAT mode
>                  (allow outgoing sessions but not incoming), but without the
>                  extra mangling rules to NAT the packets.
>
> Since the different filtering options only make sense for forward
> mode='route', it's sensible to put the option in the <forward> element.
>
> But that's really a separate topic that I didn't originally intend to
> get entwined with the dns/ra/logging things we're talking about now...
Yes, but it all does tie together.

By default, if no IP addresses are specified, then you are "restricted" 
in that no communications exists between any guests using that virtual 
interface and the virtualization host.  But, if no IP addresses are 
specified, then, as currently implemented, IPv4 communications is 
permitted between guest virtual systems.  I only wish to have this same 
capability available to IPv6.

BTW, if a user want dns, dhcp, Ra, etc. services, then it has to be 
implemented in a guest virtual system.
>
>>>> I have scratched an itch of mine by providing a way to specify that
>>>> dns-queries and dhcp activity should be logged and that this is
>>>> specified separately for each virtual network interface.
>>>>
>>>> Dan suggested that a user might still want to run dnsmasq's dns
>>>> service even if no IP addresses are specified.  However, in that case
>>>> I am not sure what dnsmasq is suppose to monitor for queries ... it
>>>> needs some kind of address or we will have the problems which the
>>>> bind-dynamic update is suppose to fix.
>>> Yeah, I didn't understand that either - If the bridge has no IP address,
>>> there's no target IP for a DNS request to be sent to. I just assumed he
>>> was thinking of something else.
>> Believe it or not, it is possible to start dnsmasq on an interface
>> which has no gateway addresses and --port=0 so that there no dns ...
>> it runs and appears to do nothing.  Again, what purpose is there to
>> starting dnsmasq in this situation?
>
> As I implied above, I don't think there is any purpose to it.
I agree.  And since we are in agreement, that is how I will implement 
it.  And, it would make sense to have:
    <dns enable='yes|no' />
>
>
>>>
>>>> I have not integrated this with the bind-dynamic update but plan to
>>>> before I submit it.  I will be submitting this patch with the IPV6
>>>> enhancement, DHCPv6, etc. series since it is yet again messing with
>>>> the same code.
>>>>
>>>> If any of the series get rejected rather than accepted, it will not be
>>>> too much of a problem to rework them.
>> One last question.  Considering all of these various new xml
>> parameters/attributes, should they all be on the <network> statement,
>> only part of them on <network> and others on a (perhaps) a <logging>
>> defintion, or maybe an <options> statement.
> If you want to consolidate options, rather than grouping according to
> <logging> and <options>, why not put the dhcp-related options in the
> <dhcp> element (edit: see below - I think I've changed my mind about
> <dhcp>), and the dns-related options in the <dns> element as I
> originally proposed. I can't think of any reason why (especially
> considering that the <dns> element already exists as a direct
> sub-element of <network> that this:
>
>   <network>
>     <dns enable='no'/>
>
> or:
>
>   <network>
>     <dns log='yes'/> <!-- or perhaps use "debug|info|warn|error|none" or
> something -->
>
> is any worse than this:
>
>     <network disableDns='yes'>
>
>    or
>
>     <network logDns='yes'>
>
> or this:
>
>     <network>
>       <option disableDns='yes'/>
>
>    or
>
>     <network>
>       <logging dns='yes'/>
>
> or even this:
>
>     <network enableDns='no'>
>
>    or
>
>     <network>
>       <option enableDns='no'/>
>
> Rather, I would argue that it is *better* to put dns-related options in
> the already-existing <dns> element. Why create a new element that mixes
> the options of different functions when an element for each of those
> functions already exists.
OK, enable= and log= will go in the <dns> element.

For, enableRA, it will go in any <ip family='ipv6">. If enableRA='yes' 
and if dhcp is specified for that element, the stateful RA will be 
configured.  If enableRA='yes' (the default) and no dhcp is specified, 
then stateless RA is configured.  If enableRA='no', no RA is configured 
for that subnet.  If all ipv6 specifications have enableRA='no', then 
nether radvd will be started nor will dnsmasq be configured for RA.

If any IPv4 or IPv6 DHCP specification which includes log='yes', then 
log-dhcp will be specified for the interface.

When (hopefully not if) dnsmasq is changed to include log-ra to enable 
messages about RA sent to syslog and log-ra is added to --help, then any 
<IP family='ipv6' logRA='yes' ... /> will enable it for all IPv6 
addresses on that interface.

If dns is disabled and there is no dhcp specification but RA is not 
disabled on all IPv6 specifications, then ??

It is not clear to me that dnsmasq could handle state-less RA if there 
is no dns and no dhcp specified.  In fact, the way state-less RA is 
specified is with dhcp-range=<ipv6-address>,ra-only ... more testing!
>
>
>> Comments? Suggesting?
> 1) I think dns should be disabled with "<dns enable='no'/>". The element
> already exists, and is the perfect place to add dns-related options
> (that is *why* it exists)
OK, for the <dns> element, enable= and log= will be added.
>
> 2) disabling IPv6 RA should be done by having <ip family='ipv6' ....
> enableRa='no' ...> in each ipv6 ip address element - if all ipv6
> addresses have enableRa='no', then radvd (or the ra in dnsmasq) won't be
> run.
Yup.
>
> 3) we don't need any option to disable dhcp - that's what happens if
> there are no <dhcp> elements.
Yup.
>
>
> 4) I think I've changed my mind about logging options - these options
> would need to be set on at least a network-wide scale, but while there
> is a perfect existing element for that for dns, the <dhcp> and <ip>
> options are one-per IP address, which would lead to ambiguity and/or
> conflict. Beyond that, there is no existing example of setting logging
> level anywhere in libvirt XML - the only logging controls are done in
> the environment or via libvirtd.conf, and I don't know that either of
> those places are appropriate either. So I think we should separate that
> off and discuss it separately.
See above ... if any log(dhcp)='yes' or any logRA='yes', that logging 
will be enabled for the interface.  As I see it, the alternative is to 
add it to <network>.

Now, a little coding and a whole bunch of testing.  And, some 
"discussion" about what/how-much dnsmasq send to syslog.

Gene