[libvirt] [RFC] OpenVSwitch datapath support (was: Driver development)

Carlos N. A. Corrêa carlos.nilton at gmail.com
Thu Jun 9 03:34:10 UTC 2011


Hi, Cole!

As I said, I gone through the list archives and got some insight on how to
present my ideas.

I tried to use the thread started by Laine Stump (
https://www.redhat.com/archives/libvir-list/2011-April/msg00591.html) as a
model.

Please, forgive me if my RFC is too verbose: I tried to produce something
good.

In time: I'm willing to implement this idea as part of my postgraduate work.


Index
=====
1. Incorporating Open vSwitch support to libvirt
2. Background
3. Architecture
3.1. Option 1
3.2. Option 2
4. Implementation

1. Incorporating Open vSwitch support to libvirt
================================================

OpenFlow (http://www.openflow.org/) is a technology from the Software
Defined Networks (SDN) fame. Switches supporting it export a software
interface that allows special programs (Controllers) to dictate how the
forwarding of ethernet frames should be done. That decision can not only be
based on traditional layer-2 information (physical address 00:01:02:03:04:aa
maps to port 1, address 05:06:07:08:09:bb maps to port 2, and so on) but
also on information from a mix of network layers, like VLAN tags, IP
addresses and/or TCP ports.

Controllers may take network flexibility and efficiency to a new level,
slicing switches' capacity transparently to run concurrent, isolated
networks (FlowVisor, https://openflow.stanford.edu/display/flowvisor/Home).
They can also incorporate logic to make the connectivity layer play a
greater role in areas like IP routing (RouteFlow,
http://sites.google.com/site/routeflow/), network access control (SMAC,
http://www.openflowhub.org/display/Snac/SNAC+Home), hosts load-balancing (
http://www.cs.princeton.edu/~jrex/papers/loadwild10.pdf) and live migration
(http://conferences.sigcomm.org/sigcomm/2008/papers/p513-ericksonA.pdf).

In a very practical view, as libvirt supports a programmatic definition of
network nodes, OpenFlow allows for a programmatic definition of traffic
forwarding. An IaaS/cloud environment composed of many physical hosts and
extensive use of libvirt for hypervisor management could use OpenFlow for
virtualized switches management.

Even tough its current libvirt enables one to define and instantiate virtual
networks, those networks are, basically, Linux kernel bridges which
configuration (device IP addresses, naming, connections) is stored into
network XML files. Linux bridges doesn't offer a programmatic interface, and
depend on not-so-easily-customizable approaches (VLANs, tunnels) to properly
isolate traffic.

Open vSwitch (or OVS, http://www.openvswitch.org/) is a full software
implementation of a network switch. It supports standard management
interfaces and protocols (e.g. NetFlow, sFlow) and is being used in a
multitude of virtual and cloud-based computing (e.g. XenServer, OpenNebula).
Even more, OVS can operate in "OpenFlow-mode".

When operating in a traditional ("full switch") fashion, an OVS instance is
similar to a Linux bridge - just supporting features that makes them work
more like a "real" switch. In this scenario, OVS kernel driver even includes
a compatibility layer that allows bridge-utils (and libvirt) to act upon OVS
instances just as if they were Linux bridges!

When operating in OpenFlow mode, the OVS instance is called a _datapath_.
Host interfaces can be bound to and send traffic through it, but forwarding
logic is provided by the Controller process. In this scenario,
OpenFlow-based solutions could be leveraged to virtual networks. Even more,
OpenFlow-based solutions which use virtual facilities (like VMs and OVS)
could use libvirt as part of its frameworks!

I'm proposing changes to libvirt's network XML format. It would allow for
"datapath" elements, and it's attributes would allow libvirt to instantiate
a OVS datapath, specifying it's controller address and hardware description,
for example.

2. Background
=============

To instantiate an OVS datapath (DP), one must call the ovs-openflowd daemon.
Some relevant aspects of its syntax are covered below:

------------------------------- 8< -------------------------------

usage: ovs-openflowd [OPTIONS] DATAPATH [CONTROLLER...]

Active OpenFlow connection methods:
  tcp:IP[:PORT]         PORT (default: 6633) at remote IP
  ssl:IP[:PORT]         SSL PORT (default: 6633) at remote IP
  unix:FILE               Unix domain socket named FILE

Passive OpenFlow connection methods:
  ptcp:[PORT][:IP]        listen to TCP PORT (default: 6633) on IP
  pssl:[PORT][:IP]        listen for SSL on PORT (default: 6633) on IP
  punix:FILE              listen on Unix domain socket FILE

OpenFlow options:
  -d, --datapath-id=ID    Use ID as the OpenFlow switch ID
                          (ID must consist of 16 hex digits)
  --mfr-desc=MFR          Identify manufacturer as MFR
  --hw-desc=HW            Identify hardware as HW
  --sw-desc=SW            Identify software as SW

Networking options:
  --out-of-band           controller connection is out-of-band

Daemon options:
  --detach                run in background as daemon
  --pidfile[=FILE]        create pidfile (default:
/usr/local/var/run/openvswitch/ovs-openflowd.pid)

------------------------------- 8< -------------------------------

An execution example can be seen below:

# ovs-openflowd --hw-desc=rfovs dp0 tcp:127.0.0.1:6633 --out-of-band
--detach

This instantiates an OVS datapath with hardware description "rfovs". The
datapath name (and Linux virtual interface name) is "dp0". The controller
software is running in the same machine as the OVS instance (IP address
127.0.0.1), and can be reached through port 6633/TCP. The connection to the
controller must not be done through the datapath itself (out-of-band), and
the ovs-openflowd will run as a background daemon.

3. Architecture
===============

libvirt documentation available on
http://www.libvirt.org/formatnetwork.htmldescribes the following
format to network definition:

<network>
        <!-- General metadata -->
        <!-- Bridging, naming and forwarding data -->
        <!-- Addressing data -->
</network>

Based solely in that documentation, the format is somewhat bound to Linux
bridges. This is not a problem most of the time, as OVS includes a
compatibility module (brcompat_mod) that allows bridge-utils to configure an
OVS "full switch" just as if it was a traditional Linux bridge. The syscalls
offered also permit libvirt to manage these instances transparently.

However, this format doesn't provide sufficient information to configure a
OVS datapath. Next, I propose some options of how the libvirt network format
could be changed to provide this facility (and, off course, to implement
those changes).

3.1. Option 1: network type
---------------------------

I saw this in an e-mail from Laine Stump in last April (
https://www.redhat.com/archives/libvir-list/2011-April/msg00640.html). It
seems like my preferred ideas about how the change could be accomplished.

The network opening element would be extended to include an optional "type"
attribute. It could be defined as "bridge" (the default behaviour, for
backwards compatibility) or "ovs-datapath" (the case for which I would like
to work):



<network type="bridge"> (THE DEFAULT)

or

<network type="ovs-datapath">



As the "General metadata" from the network format is instrumental to
libvirt, it would be necessary to a "ovs-datapath" network.

As the control layer of an OVS datapath lie outside itself, the "Bridging,
naming..." section of the network format would be replaced by the following
elements/attributes:



<datapath name="" id="" mfrdesc="" hwdesc="" swdesc="">

    - Mandatory, this element would allow for datapath and OpenFlow general
configuration.
    - The name attribute would be used to designate the datapath virtual
interface. (Mandatory)
    - The id, mfrdesc, hwdesc and swdesc attributes would be mapped 1:1 to
ovs-openflowd parameters presented before. (Optional)

<controller protocol="" address="" port="" outofband="">
OR
<controller protocol="" path="">

    - Protocol. Would be one of "tcp" or "unix", by now. (Mandatory)
    - Address. Would be an IP address. (Mandatory, if protocol="tcp")
    - Path. Would be the path to a UNIX socket to which the controller
process is listening. (Mandatory, if protocol="unix")
    - Port. Would be the port to which the controller is listening to.
(Optional, default = 6633)
    - Outofband. If connection to the controller must be done "in" or "out"
of band. (Optional, default = "NO").
    - The controller element itself would be optional, as OVS includes a
"controller discovery" process by default. If specified, however, it's
mandatory attributes should be respected.



For the sake of a proof-of-concept implementation, this would be the entire
(!) scope of the initial change/implementation. After this step, things such
as this could be added:

- An "ssl" option for the "protocol=" attribute, to allow for SSL-encrypted
connections between datapaths and its controllers.
- An element to pass SSL connection data (private key, ca cert, etc.) to
ovs-openflowd. E.g.: <pki privatekey...>
- Elements to control networking operation (e.g. max-idle, max-backoff) and
rate-limiting of the datapath.

3.2. Option 2: datapath element
-------------------------------

In place of a network type definition, just a new "datapath" element would
be added. The network_conf code should test if a "bridge" element is
specified in a network definition, and look for a datapath if it can't find
one. Addressing data found in a network using the "datapath" element would
be ignored.

The datapath element here could be overly long, including description,
controller and ssl-related attributes - but the format change would be very
minimal:



<network>
        <!-- General metadata -->
        <datapath name="" descriptions="..." controller_data="...">
</network>



Other option would be set those properties as sub-elements of <datapath...>,
merging this option with the first:



<network>
        <!-- General metadata -->
        <datapath name="" id="" mfrdesc="" hwdesc="" swdesc="">
            <controller protocol="" address="" port="" outofband="">
            <pki privatekey...>
            <networking...>
        </datapath>
</network>



Besides being and option, this scenario (in fact, the entire option 2) seems
somewhat unclean to me.

4. Implementation
=================

1. I can hack through the network_conf code to make it parse the XML. That's
an easy part.
2. I can use the bridge driver as a model + your command execution
infrastructure (which is very cool, IMHO) to implement an OVS driver.
3. It's not yet very clear to me where is the code that instantiates each
network after it's "def" get parsed. Well, after the first two steps, I
think that I would need to just hook the parsing and driver functions
together in instantiating/virsh/command-line code.

That's all, folks!

Best regards,
Carlos "Bill" Nilton
CISSP, RHCE, ITIL Foundations
http://carlosnilton.com.br/



Em 8 de junho de 2011 17:34, Carlos N. A. Corrêa
<carlos.nilton at gmail.com>escreveu:

> Hi,
>
> I gathered information about that and I'll produce relevant documentation
> of my point tonight.
>
> I'll be glad to do an informed and well structured work. Thanks!
>
>
> Best regards,
> Carlos "Bill" Nilton
> CISSP, RHCE, ITIL Foundations
> http://carlosnilton.com.br/
>
>
>
> Em 8 de junho de 2011 15:41, Cole Robinson <crobinso at redhat.com> escreveu:
>
> On 06/08/2011 07:10 AM, Carlos N. A. Corręa wrote:
>> > Hi,
>> >
>> > I'm interested in developing a new network driver for libvirt (one for
>> the
>> > definition and management of OpenVSwitch datapaths).
>> >
>> > I've read some of the code already, and by now my general directions
>> are:
>> >
>> > - Change the function virNetworkDefParseXML in src/conf/network_conf.c
>> to
>> > define and parse new configuration directives needed by my driver
>> > - Use src/network/bridge_driver.c as a template for my new driver,
>> replacing
>> > code from the services provided by the driver (listed on
>> "virNetworkDriver"
>> > type) with my own code, for my own net type.
>> >
>> > Have you any other directions in this matter? Any tips? If it works,
>> would
>> > be this code of any use to you?
>> >
>>
>> For a starting point I'd recommend proposing your XML changes to the
>> list, and describe exactly what the new functionality will enable, and a
>> few use cases. Probably best to do this before starting any code so you
>> don't head in a wrong direction.
>>
>> Check the past few month ML archives to see how some other developers
>> have proposed RFCs or XML changes.
>>
>> - Cole
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20110609/4743ecde/attachment-0001.htm>


More information about the libvir-list mailing list