[libvirt] RFC: configuring host interfaces with libvirt

Mon Feb 2 09:59:02 UTC 2009

On Thu, Jan 22, 2009 at 03:39:41PM -0700, Jim Fehlig wrote:
> Adding Marius (SuSE sysconfig maintainer) in case I'm misspeaking :-)
> 
> David Lutterkort wrote:
> >On Tue, 2009-01-20 at 11:35 -0700, Jim Fehlig wrote:
> >  
> >>David Lutterkort wrote:
> >>    
> >>>For certain applications, we want libvirt to be able to configure host
> >>>network interfaces in a variety of ways; currently, we are most
> >>>interested in teaching libvirt how to set up ordinary ethernet
> >>>interfaces, bridges, bonding and vlan's.
> >>>
> >>>Below is a high-level proposal of how that could be done. Please comment
> >>>copiously ;)
> >>>
> >>>1. XML Format
> >>>=============
> >>>
> >>>The first question is how the user would describe the host interfaces 
> >>>they
> >>>want. Below are some sketches of what an XML representation of the 
> >>>various
> >>>kinds of interfaces would look like. This covers the minimal amount of
> >>>settings for these to be useful, though I am sure we'll need to add more
> >>>over time.
> >>>
> >>><interface device="eth0" onboot="yes">
> >>>  <hwaddr>00:19:d2:9f:88:96</hwaddr>
> >>>  <dhcp peerdns="yes"/>
> >>></interface>
> >>>  
> >>>      
> >>For onboot="yes", something like startmode="onboot" would be better
> >>IMO.  A startmode attribute would allow also using "manual", "hotplug",
> >>"ifplug", "nfsroot", etc.
> >>    
> >
> >This makes only sense if these additional modes are completely
> >orthogonal; I am not sure what the start modes 'manual', 'ifplug', and
> >'nfsroot' should do.
> 
> manual
>  Interface will be set up if ifup is called manually.
> ifplugd
>  Interface will be controlled from  ifplugd
> nfsroot
>  Nearly like auto, but interfaces with this startmode will never be
>  shut down via rcnetwork stop

On SUSE we have:
	auto == on == onboot:
		Interface will be set up as soon as it is available
		and service network was started.
	hotplug:
		The difference between auto and hotplug is that the
		latter does not make rcnetwork fail (or wait for it)
		if the interface cannot be brought up.

> > For onboot, I assume you mean that they translate
> >into assignments in ifcfg in the following way
> >
> >        no startmode attribute  -> ONBOOT=no
> >        startmode='onboot'      -> ONBOOT=yes
> >        startmode='hotplug'     -> ONBOOT=yes and HOTPLUG=yes

On SUSE there is a STARTMODE variable instead of separate ONBOOT,
HOTPLUG, ...

> On SuSE, the related ifcfg syntax is
> 
> STARTMODE={manual|auto|hotplug|ifplugd|nfsroot|off}
> 
> >>WRT dhcp element, would it make sense to consider hostname (hostname to
> >>send to server) and if to change the local hostname to the hostname
> >>delivered via dhcp?  E.g.
> >>    hostname="foo" (default `hostname`)
> >>    sethostname
> >>
> >>Also route:
> >>    setrouting (default "yes")
> >>    setdefaultroute (default "yes")
> >>
> >>and NIS:
> >>    nis (default "yes")
> >>    setnisdomain (default "yes")
> >>    
> >
> >Yes, they all make sense, though I would probably not support them in
> >the very first cut.
> >
> >  
> >>What about dhcpv6?  A separate <dhcp6 /> element?
> >>    
> >
> >Yeah, we probably need a separate element for that since it pulls in a
> >slew of other config options.

Well, what I mean is just, that it IMO makes sense to consider it as far
as possible in the design.

> >>><interface device="eth1" onboot="yes">
> >>>  <hwaddr>00:19:d2:9f:88:97</hwaddr>
> >>>  <static ipaddr="192.168.0.5" gateway="192.168.0.1" 
> >>>  netmask="255.255.255.0"/>
> >>></interface>
> >>>  
> >>>      
> >>Similarly, support for IPv6 here.  Either 2 formats:
> >>
> >>ipaddr="192.168.0.5" netmask="255.255.255.0"
> >>ip6addr="2001:DB8:C::5/64"
> >>
> >>or allow /len for both:
> >>
> >>ipaddr="192.168.0.5/24"
> >>ipaddr="2001:DB8:ABCD::1/64"
> >>    
> >
> >Looks a little strange, but it would simplify notation.

We're using IPADDR[suffix] = {IPv4Addr|IPv6Addr}[/prefixlength] on 
SUSE; the notation with NETMASK, BROADCAST or separate PREFIXLEN
is still supported, but used as fallback only.

> >>As for routing info, how about a separate route element:
> >>
> >><route gateway="192.168.0.1" /> # destination=default
> >><route destination="default" gateway="192.168.0.1" />
> >><route destination="10.0.0.0/8" gateway="192.168.0.2" />
> >><route destination="2001:DB8:C::/64" gateway="2001:DB8:C::1" />
> >><route destination="2001:DB8::/32"> # unrecheable route (loopback)
> >>
> >>It would perhaps make sense to use iproute2 names, that is prefix
> >>instead of destination and nexthop instead of gateway.
> >>    
> >
> >For now, I want to stay out of setting up static routes, but I think
> >that has to come sooner or later.

It is OK to limit routes to the default route for now, but IMO
it is better to use separate xml nodes, e.g.:

    <static ipaddr="192.168.0.5" [netmask,broadcast,...] />
and something like:
    <route gateway="192.168.0.1" /> # implicit destination=default
    <route destination="default" gateway="192.168.0.1" />

rather than mixing the gateway into the IP address related attributes:

    <static ipaddr="192.168.0.5" gateway="192.168.0.1"
            netmask="255.255.255.0"/>

because as soon as you start to support static routes, there are two
nodes/places where the default gateway can be defined.

> >>Don't recall if this was already brought up, but need a way to specify
> >>MTU and perhaps ethtool settings as well.
> >>    
> >
> >Yeah, that is definitely needed.
> >
> >  
> >>><interface device="eth2" onboot="yes">
> >>>  <hwaddr>00:19:d2:9f:88:98</hwaddr>
> >>></interface>
> >>>
> >>><bond name="bond00" onboot="yes" mode="active-backup">
> >>>  <slave device="eth0" primary="yes"/>
> >>>  <slave device="eth1"/>
> >>></bond>
> >>>  
> >>>      
> >>In addition to mode attribute for bonds, support for miimon,
> >>arp_interval, and arp_ip_target?
> >>    
> >
> >Sure .. if the initscript suports it, it's easy enough to do ;)

On SUSE it does... ;-) This are critical parameters and every bond
interface has to use either miimon (default) or the arp_interval +
arp_ip_target combination or serious network degradation will occur
during link failures.

> >>><bridge name="br0" stp="off" onboot="yes">
> >>>  <member device="eth2"/>
> >>>  <dhcp peerdns="yes"/>
> >>></bridge>
> >>>  
> >>>      
> >>Attributes to support hellotime, forwarddelay, and maxage?  This would 
> >>allow
> >>    <bridge name="br0" stp="on" hellotime="1" maxage="4" fowarddelay="4"
> >>..>
> >>    
> >
> >My main concern with these is that only forwarddelay, stp, and gcint are
> >controllable by Fedora/RHEL networking scripts. For hellotime and
> >maxage, we'd need to hook into the various ifup scripts atthe right
> >place. Are these directly supported by the SuSe infrastructure ?
> >  
> 
> This is probably best answered by Marius.

Yes, we support all bridge parameters in SUSE ifcfg files.

The problem occurs as soon as STP is on, because a bridge using the
IEEE default settings, needs 50 seconds to become fully operational
and no traffic goes through in this time... so there is a need to
tune at least the hellotime, maxage, and fowarddelay or all sort of
another timeout settings (e.g. dhcp).
See http://tldp.org/HOWTO/BRIDGE-STP-HOWTO/advanced-bridge.html

Another way would be to say, there is either no STP parameter at all
(and use always stp="off" + fowarddelay=0) or only the STP parameter
and the backend implementation has to handle the another parameters
and write them "using real world defaults" into the ifcfg file.

Gruesse / Regards,
 Marius Tomaschewski <mt at suse.de>
--
 Server Technologies Team, SUSE LINUX Products GmbH,
 Nuernberg; GF: Markus Rex; HRB 16746 (AG Nuernberg)
 PGP public key on:    http://www.suse.de/~mt/mt.pgp
 DF17 271A AD15 006A 5BB9   6C96 CA2F F3F7 373A 1CC0