[libvirt] Proposal: no dnsmasq (no dhcp and no dns) and no radvd option

[Laine Stump]

On 11/25/2012 08:25 AM, Gene Czarcinski wrote:
> On 11/24/2012 04:19 PM, Laine Stump wrote:
>> On 11/23/2012 02:18 PM, Gene Czarcinski wrote:
>>> On 11/20/2012 05:29 PM, Laine Stump wrote:
>>>> On 11/20/2012 02:36 PM, Gene Czarcinski wrote:
>>>>>> Laine mentioned something yesterday that got me to thinking: being
>>>>>> able to specify that dnsmasq is not to be started for an interface.
>>>>>>
>>>>>> Let me expand that by saying that libvirt would not start dnsmasq
>>>>>> for
>>>>>> either dns or dhcp and also would not start radvd.  However, the
>>>>>> IPv4
>>>>>> and IPv6 gateway addresses would be defined on the virtual network
>>>>>> interface and the "usual" iptables and ip6tables rules would be in
>>>>> force.
>>>>>> This would allow a user to configure dnsmasq to meet any user
>>>>>> desires
>>>>>> or use something completely different instead of dnsmasq.
>>>>>>
>>>>>> Questions:  Useful?  Worth the time and effort?
>>>> That was already determined before I mentioned it to you - it's been
>>>> requested several times, and I've told some people it was "going to
>>>> happen", although didn't say when:-).
>>>>
>>> OK, color this "almost done" (doc and schemas need updating).
>>>
>>> Three new parameters are added:
>>>        <network disableDnsmasq='yes'  logDnsQueries='yes'
>>> logDhcp='yes' >
>> Don't refer to dnsmasq specifically in the XML unless it's something
>> that is unique to dnsmasq (and then we should seriously consider  if
>> we're thinking about it the right way before adding it). The intent is
>> that the XML should describe the desired configuration in a manner high
>> level enough that a new driver using different backend components would
>> be able to use the same XML attributes and arrive at similar
>> functionality. For example, there is active work right now to support
>> libvirt's server side on FreeBSD, and they may decide that they want to
>> implement their networks using different implementations of DNS and dhcp
>> servers (they will already need to do something different in place of
>> Linux host brdges and iptables, so it will have to be a completely new
>> driver anyway).
>>
>> Obviously there are cases where something specific to the backend needs
>> to be in the XML; for example the virtio-net drivers can specify <driver
>> name='qemu' ...> and in that case can have an option ioeventfd and
>> event_idx options which would probably be meaningless to any netdev
>> driver for another hypervisor, but we aim to keep that to a minimum, and
>> use it only when the characteristic described really is unique to
>> particular backend).
>>
>> (At any rate, it seems reasonable to me that someone may want us to
>> provide a dhcp server but no dns server. Yes, this also implies that we
>> should have a way in the config to feed in the address of an alternative
>> DNS server ("--dhcp-option=option:dns-server,x.x.x.x" and
>> "--dhcp-option=option6:dnsserver,xxxx::xxxx") to send to dhcp clients
>> when they've disabled dnsmasq's own dns server (in case you hadn't found
>> it in the manpage yet - you can run dnsmasq with no dns server by adding
>> "--port=0" to the commandline). As soon as I hit send on this, I'm going
>> to type up a short mail asking for comments on how to add support for
>> any standard dhcp option.
>>
>> As for the other two - any reason why you used "logDnsQueries" instead
>> of "logDns"? Also, do you think we might want the ability to specify a
>> level in the future? (I guess if we did, that could coexist with
>> yes|no).
>>
>> And while we're on the topic of logging: one thing that's bothered me a
>> lot, and we've had other people complain about it as well, is that every
>> single DHCP lease renewal results in a message to /var/log/messages, and
>> there doesn't seem to be any way to suppress it. Is that correct?
>
> OK, I had not realized that it was possible to not have dns while
> continuing to do dhcp.  So, it now becomes disableDns="yes" which
> results in --port=0 being passed to dnsmasq.  For completeness, I
> suggest that disableDhcp="yes" and disableRA="yes" be added.

disableDhcp already exists - you just don't have a <dhcp> section (and
if you do have a <dhcp> section, it seems very illogical that you would
want dhcp disabled).

disableRA may be a valid concern, since that's on by default if there
are any IPv6 addresses.


However, to pull the discussion back in - I had earlier suggested *not*
having an attribute called "disableXXX" because it starts to get
confusing - you're saying "Yes, I don't want this feature.". It's
simpler to understand "*No*, I *don't* want this feature". And I'd also
suggested making it an option of the already existing <dns> element (see
below).

>
> I have changed my mind and will make it logDns=, logDhcp= and (again
> for completeness) logRA= ... this last one may require a small dnsmasq
> update.

I like "<dns log='xxx'.../>" (edit: I may have changed my opinion on
this - see the end of the message)

>
> However, I still like the idea of not starting anything and, as I
> state below, you can start dnsmasq even if it is configured to do nothing.

Why? That would only make sense if it was a placeholder that could be
modified later to do something (and even then only if it would be
impossible to just start it at a later time instead).

>   Therefore, I propose adding "disableDaemons=" which is not specific
> to any package but would mean not to start either dnsmasq or radvd
> regardless of whether or not they would be started.

I dislike that idea - it give two ways to specify the same thing, which
is bad. Experience has shown that it's better to have a single way to
specify each thing; that way there is no ambiguity when doing a
parse->format->parse cycle.

Remember that you can easily have no dnsmasq daemon by simply specifying:

   <dns enable='no'>

and then having no <dhcp> element anywhere.

As for RA, that will require an option to turn off. We currently only
start radvd if networkRadvdConfContents returns a non-NULL string, and
it is *perfectly* setup to have an "enableRa='yes|no'" (with default of
'yes') on each IP address - it would simply not append the lines for an
IP address to the conffile string if that IP's enableRa was no. So, if
all the ipv6 addresses had enableRa='no', radvd wouldn't be run (and I'm
sure you could do something similar for the ra implementation in dnsmasq).


>
> Lease renewal messages should not be that frequent it is probably a
> "small matter of programming" to get an option to suppress them ... if
> Simon Kelley would add that.

An indicator that something more than just log='yes|no' may be desired.

>
> Speaking of logging, if you do not like the lease renewal messages,
> then you will have a bigger problem with the "RTR-ADVERT" messages
> that dnsmasq issues every 3//4/5 seconds for every started interface.

Yes, that's definitely a no-go. I'm pretty sure nobody would stand for
that. If that's how dnsmasq operates, we can't switch to using dnsmasq
until that's remedied.

>   I have submitted a "tiny" patch to dnsmasq which ties issuing this
> message to log-dhcp being enabled.


That will be good, but since it will be something not detectable from
the --help output, it will mean that use of dnsmasq for RA will need to
be dependent on an "even later" version of dnsmasq.


>>
>>> If nothing is specified, then the default is no, no, no so that things
>>> work as they do now.  Currently, if the boolean value is 'no' or
>>> false, then that parameter is not written out.  I could easily change
>>> that depending on what others want to do.
>>>
>>> Laine, I changed this to be disableDnsmasq because that is the real
>>> result.  If this is specified, then starting radvd will also be
>>> suppressed.  It is as if no IPv4 or IPv6 addresses were specified.
>>> However, the iptables and ip6tables rules will remain the same since
>>> it is assumed that gateway addresses will be specified.
>> Actually that's yet another switch that we need to add - the ability to
>> suppress all iptables rules for a particular network (unless it is in
>> NAT mode, in which case it makes no sense, because it's the difference
>> in rules that changes it from mode='route' to mode='nat'). Perhaps it
>> could be something like filter='none|open|restricted', although I don't
>> know if that's descriptive enough - even without getting into specific
>> protocols (which I don't think should be done - at that point they
>> should really be using a guest-specific filterref) you might still want
>> to know if there should be any rules at all, allow everything, allow
>> outgoing sessions but not incoming, allow inter-guest traffic but not
>> incoming from the outside, allow nothing (all of these could be
>> overridden by a particular guest's filterref)
> Easy enough to implement but what is the purpose.  If there are no
> rules at all (and perhaps no gateway addresses), then the interface
> will not work for guest-to-guest communications either.

Because some people want libvirt-created networks, but want to manage
the firewall rules themselves.

(btw, you must be running with net.bridge.bridge-nf-call-iptables=1,
otherwise communications between guests (ipv4 and ipv6) would work just
fine with no extra rules)

> But wait, if there are no rules at all the the "default"
> "--reject-with icmp-port-unreachable" will not be there either so
> maybe things will work.  However, I thought that filters were
> specified as part of the <interface> definition, not the <network>
> definition.

Yes, exactly. The network would have no rules of its own (or maybe a
ruleset to allow only incoming, as is currently the case for a NATed
network), and each guest's interface could override that with its own
rules to allow incoming.

>
> So, where should this be handled?  It makes some sense to me that for
> the types of networks src/network/bridge_driver.c handles, maybe
> <network> is more appropriate.

More appropriate than what? Than using a <filterref> in each guest's
interface? Or more appropriate than making it an option of <forward>? I
think it makes sense to have this be an option of <forward>, because it
is controlling exactly what traffic is allowed to be forwarded. And when
you look at the existing forwarding modes:

<forward mode='nat'>

It doesn't make sense to have different levels of filtering for a NATed
network, because the filter rules are required for NAT to work at all,
and due to the way NAT works, incoming sessions are impractical anyway:

   none - nonsensical (if you're not going to allow any traffic in-out,
          then why have <forward> at all?)
   open - nonsensical (incoming sessions can't work without additional
          forwarding rules anyway)
   restricted - that's what it does anyway

<forward mode='bridge|hostdev'>

These networks have nothing managed by libvirt anyway

<forward mode='none'> (i.e. "no <forward> element)

By definition if you have this type of network, you don't want to allow
any traffic in or out of the network (heck, we even run dnsmasq with
different options so that the dns server won't forward requests for
unknown hosts).

<forward mode='route'>

This is the *only* type of forwarding where it makes sense to offer
different options for the stock filters provided by libvirt.

   none - the host admin will be managing filters themselves external to
libvirt.

   open - this is what it currently does - allows incoming and outgoing
sessions

   restricted - this would be similar to what is currently done in NAT mode
                (allow outgoing sessions but not incoming), but without the
                extra mangling rules to NAT the packets.

Since the different filtering options only make sense for forward
mode='route', it's sensible to put the option in the <forward> element.

But that's really a separate topic that I didn't originally intend to
get entwined with the dns/ra/logging things we're talking about now...

>>
>>> I have scratched an itch of mine by providing a way to specify that
>>> dns-queries and dhcp activity should be logged and that this is
>>> specified separately for each virtual network interface.
>>>
>>> Dan suggested that a user might still want to run dnsmasq's dns
>>> service even if no IP addresses are specified.  However, in that case
>>> I am not sure what dnsmasq is suppose to monitor for queries ... it
>>> needs some kind of address or we will have the problems which the
>>> bind-dynamic update is suppose to fix.
>>
>> Yeah, I didn't understand that either - If the bridge has no IP address,
>> there's no target IP for a DNS request to be sent to. I just assumed he
>> was thinking of something else.
> Believe it or not, it is possible to start dnsmasq on an interface
> which has no gateway addresses and --port=0 so that there no dns ...
> it runs and appears to do nothing.  Again, what purpose is there to
> starting dnsmasq in this situation?


As I implied above, I don't think there is any purpose to it.


>>
>>
>>> I have not integrated this with the bind-dynamic update but plan to
>>> before I submit it.  I will be submitting this patch with the IPV6
>>> enhancement, DHCPv6, etc. series since it is yet again messing with
>>> the same code.
>>>
>>> If any of the series get rejected rather than accepted, it will not be
>>> too much of a problem to rework them.
>>
> One last question.  Considering all of these various new xml
> parameters/attributes, should they all be on the <network> statement,
> only part of them on <network> and others on a (perhaps) a <logging>
> defintion, or maybe an <options> statement.

If you want to consolidate options, rather than grouping according to
<logging> and <options>, why not put the dhcp-related options in the
<dhcp> element (edit: see below - I think I've changed my mind about
<dhcp>), and the dns-related options in the <dns> element as I
originally proposed. I can't think of any reason why (especially
considering that the <dns> element already exists as a direct
sub-element of <network> that this:

 <network>
   <dns enable='no'/>

or:

 <network>
   <dns log='yes'/> <!-- or perhaps use "debug|info|warn|error|none" or
something -->

is any worse than this:

   <network disableDns='yes'>

  or

   <network logDns='yes'>

or this:

   <network>
     <option disableDns='yes'/>

  or

   <network>
     <logging dns='yes'/>

or even this:

   <network enableDns='no'>

  or

   <network>
     <option enableDns='no'/>

Rather, I would argue that it is *better* to put dns-related options in
the already-existing <dns> element. Why create a new element that mixes
the options of different functions when an element for each of those
functions already exists.


> Comments? Suggesting?

1) I think dns should be disabled with "<dns enable='no'/>". The element
already exists, and is the perfect place to add dns-related options
(that is *why* it exists)

2) disabling IPv6 RA should be done by having <ip family='ipv6' ....
enableRa='no' ...> in each ipv6 ip address element - if all ipv6
addresses have enableRa='no', then radvd (or the ra in dnsmasq) won't be
run.

3) we don't need any option to disable dhcp - that's what happens if
there are no <dhcp> elements.


4) I think I've changed my mind about logging options - these options
would need to be set on at least a network-wide scale, but while there
is a perfect existing element for that for dns, the <dhcp> and <ip>
options are one-per IP address, which would lead to ambiguity and/or
conflict. Beyond that, there is no existing example of setting logging
level anywhere in libvirt XML - the only logging controls are done in
the environment or via libvirtd.conf, and I don't know that either of
those places are appropriate either. So I think we should separate that
off and discuss it separately.