[libvirt] openvswitch vlan support when not a forwarded network

[Ajith Antony]

Thank you Laine, for your prompt reply.  Adding the dev list to broaden the
audience.

On Mon, Sep 16, 2013 at 9:21 AM, Laine Stump <laine at laine.org> wrote:
On 09/16/2013 07:32 AM, Ajith Antony wrote:
>> Hi, I'm not sure what the best way to pursue this is, but since it
>> appears you authored the relevant code, you could advise of a good
>> next step.
>
> Kind of. I think I wrote code for hostdev networks, but the original was
> (if I remember correctly) written by Kyle Mestery with support only for
> openvswitch.
>
>> I'm trying to use libvirt to set up openvswitch vlans, but ran into a
>> problem where the libvirt code is precluding using vlans if the
>> network is not "Forwarded."[1]   I don't think there is any technical
>> reason not to support vlans for non-forwarded networks.   I verified
>> this by manually adding vlan tags to the ephemeral ports created by
>> the libvirt network definition.  The tags worked as expected.
>> [1] http://libvirt.org/git/?p=libvirt.git;a=blob;f=src/network/bridge_driver.c;h=3a8be90f6c76f8fb44d0d5411e65ac8663eb7771;hb=HEAD#l2416
>
> You're misunderstanding the intent (at least) of that if condition. The
> way that you configure a libvirt network to use openvswitch is with the
> following:
>
> <forward mode='bridge'/>
> <virtualport type='openvswitch'>
>    <some other stuff I forget/>
> </virtualport>
>
> That condition is only checking if:
>
>     vlanAllowed = ((def->forward.type == VIR_NETWORK_FORWARD_BRIDGE &&
>
>
> 1) forward mode is 'bridge' (this doesn't necessarily mean that anything
> will be forwarded beyond the bridge that the guest's network device is
> connected to, only that it is connected to a bridge, i.e. as far as
> libvirt is concerned any forwarding that *is* done will be L2
> forwarding. The specifics of how that happens are left up to the bridge.)

Yes, I think this the test that interests me.  The key fact in my scenario is
that I'm not preconfiguring a bridge.  I wouldn't be able to set the forward
mode as 'bridge' unless I already prepared a bridge.  I recongize that if I
prepared a bridge that I could add more portgroups to it with thier unique vlan
tags and not expect anything to actually cross the physical interface.  I think
this is what is is reccomended among the blogs when one searches for
"openvswitch fake bridge."


> You'll need to post a copy of your network config for me to understand
> for sure, but I'm guessing that you're missing one of the two items in
> your config, possibly indicating that you've misunderstood how to
> configure a libvirt network that uses an openvswitch bridge.
>
> So can you send a copy of your network XML?


The network that I'm trying to define is described as the following.  Note
there there is no <forward> element as I'm not expcting there to be a *real*
bridge and I don't expect that I need to preconfigre explicit bridges.  I
expect an ephemeral virtual bridge to be created.

    <network>
      <name>ajith-net1</name>
      <virtualport type='openvswitch'/>
      <portgroup name='client_lan'>
           <vlan>
                 <tag id='1'/>
           </vlan>
      </portgroup>
      <portgroup name='client_wan'>
           <vlan>
                 <tag id='2'/>
           </vlan>
      </portgroup>
    </network>

When I include the VLAN tags I get the error:

    error: unsupported configuration: <vlan> element specified for network
       ajith-net1, whose type doesn't support vlan configuration


When I omit the VLAN tags as follows I am able to create the network:

    <network>
      <name>ajith-net1</name>
      <virtualport type='openvswitch'/>
      <portgroup name='client_lan'>
      </portgroup>
      <portgroup name='client_wan'>
      </portgroup>
    </network>

In my domain configs I specify the network like so:

    <interface type='network'>
      <source network='ajith-net1' portgroup='client_lan'/>
      <virtualport type='openvswitch'/>
      <model type='virtio'/>
    </interface>

The resulting ephemeral bridge(virbr1) looks like the following when my
network(w/o vlans) and two domains are started. I don't know if the portgroup
was meaningful, but it was accepted in the definition:

    $ sudo ovs-vsctl show
    <...>
        Bridge "virbr1"
            Port "vnet23"
                Interface "vnet23"
            Port "vnet25"
                Interface "vnet25"
            Port "virbr1"
                Interface "virbr1"
                    type: internal
            Port "virbr1-nic"
                Interface "virbr1-nic"
        ovs_version: "1.9.3"

My goal is assign unique vlan tags to the two portgroups.   While the network
and the domains are started, I can manually add tags to the ports:

    $ sudo ovs-vsctl set port vnet25 tag=2
    $ sudo ovs-vsctl set port vnet23 tag=1
    $ sudo ovs-vsctl show
        Bridge "virbr1"
            Port "vnet23"
                tag: 1
                Interface "vnet23"
            Port "vnet25"
                tag: 2
                Interface "vnet25"
            Port "virbr1"
                Interface "virbr1"
                    type: internal
            Port "virbr1-nic"
                Interface "virbr1-nic"
        ovs_version: "1.9.3"


This configuration behaves exactly as I want.  I'm interested to learn how to
prepare this without the manual step of tagging the ports outside of the
libvirt operations.

Ultimately my goal is to prepare isolated test environments that consist of
several VM's attached to a similar qty of vlans.  I intend to create many of
these environments per host.  I also recongnize that instead of portgroups, I
could use separate networks altogether. From an administrative standpoint, I'd
prefer to have one "network" per test environment, with several portgroups,
instead of *many* networks.

FWIW, I'm doing something similar on ESX, where I create a "vSwitch" and then
create several "portgroups", each with its own vlan tag.  These vSwitches are
not attached to any physical interfaces.  (My esx work is directly against the
vsphere API, so I'm not sure if the same could be configured via libvirt)   I'm
looking to port this work to KVM.  It seems like a pretty natural thing to
support.