[libvirt] openvswitch vlan support when not a forwarded network
Laine Stump
laine at laine.org
Tue Sep 17 09:15:44 UTC 2013
On 09/16/2013 07:34 PM, Ajith Antony wrote:
> Thank you Laine, for your prompt reply. Adding the dev list to broaden the
> audience.
>
> On Mon, Sep 16, 2013 at 9:21 AM, Laine Stump <laine at laine.org> wrote:
> On 09/16/2013 07:32 AM, Ajith Antony wrote:
>>> Hi, I'm not sure what the best way to pursue this is, but since it
>>> appears you authored the relevant code, you could advise of a good
>>> next step.
>> Kind of. I think I wrote code for hostdev networks, but the original was
>> (if I remember correctly) written by Kyle Mestery with support only for
>> openvswitch.
>>
>>> I'm trying to use libvirt to set up openvswitch vlans, but ran into a
>>> problem where the libvirt code is precluding using vlans if the
>>> network is not "Forwarded."[1] I don't think there is any technical
>>> reason not to support vlans for non-forwarded networks. I verified
>>> this by manually adding vlan tags to the ephemeral ports created by
>>> the libvirt network definition. The tags worked as expected.
>>> [1] http://libvirt.org/git/?p=libvirt.git;a=blob;f=src/network/bridge_driver.c;h=3a8be90f6c76f8fb44d0d5411e65ac8663eb7771;hb=HEAD#l2416
>> You're misunderstanding the intent (at least) of that if condition. The
>> way that you configure a libvirt network to use openvswitch is with the
>> following:
>>
>> <forward mode='bridge'/>
>> <virtualport type='openvswitch'>
>> <some other stuff I forget/>
>> </virtualport>
>>
>> That condition is only checking if:
>>
>> vlanAllowed = ((def->forward.type == VIR_NETWORK_FORWARD_BRIDGE &&
>>
>>
>> 1) forward mode is 'bridge' (this doesn't necessarily mean that anything
>> will be forwarded beyond the bridge that the guest's network device is
>> connected to, only that it is connected to a bridge, i.e. as far as
>> libvirt is concerned any forwarding that *is* done will be L2
>> forwarding. The specifics of how that happens are left up to the bridge.)
> Yes, I think this the test that interests me. The key fact in my scenario is
> that I'm not preconfiguring a bridge. I wouldn't be able to set the forward
> mode as 'bridge' unless I already prepared a bridge. I recongize that if I
> prepared a bridge that I could add more portgroups to it with thier unique vlan
> tags and not expect anything to actually cross the physical interface. I think
> this is what is is reccomended among the blogs when one searches for
> "openvswitch fake bridge."
1) libvirt will not create an openvswitch bridge for you. It assumes
that you have already created an openvswitch bridge outside of libvirt.
2) As far as I understand from what I've heard of the "openvswitch fake
bridge", you *do not* want one. My recollection is that people were
using that as some method of having multiple vlans on the same bridge
when the management software provided no method of setting vlan tags.
But libvirt *does* provide a method of setting the vlan tag.
>
>
>> You'll need to post a copy of your network config for me to understand
>> for sure, but I'm guessing that you're missing one of the two items in
>> your config, possibly indicating that you've misunderstood how to
>> configure a libvirt network that uses an openvswitch bridge.
>>
>> So can you send a copy of your network XML?
>
> The network that I'm trying to define is described as the following. Note
> there there is no <forward> element as I'm not expcting there to be a *real*
> bridge and I don't expect that I need to preconfigre explicit bridges. I
> expect an ephemeral virtual bridge to be created.
What you want to do makes sense, but it's not currently possible. If you
want to use an openvswitch bridge, you need to create the bridge outside
of libvirt, then create a network with <forward mode='bridge'/> and a
<virtualport type='openvswitch'>
>
> <network>
> <name>ajith-net1</name>
> <virtualport type='openvswitch'/>
> <portgroup name='client_lan'>
> <vlan>
> <tag id='1'/>
> </vlan>
> </portgroup>
> <portgroup name='client_wan'>
> <vlan>
> <tag id='2'/>
> </vlan>
> </portgroup>
> </network>
>
> When I include the VLAN tags I get the error:
>
> error: unsupported configuration: <vlan> element specified for network
> ajith-net1, whose type doesn't support vlan configuration
for the reasons I give above. libvirt supports networks based on a
pre-existing openvswitch bridge, but isn't able to create such a bridge
itself.
That is a feature that I've contemplated though, and even tried
persuading a couple other people to write. Basically what you would need
to do is:
1) add new functions to src/util/virnetdevopenvswitch.c:
virNetDevOpenvswitchCreate()
virNetDevOpenvswitchSetSTPDelay()
virNetDevOpenvswitchSetSTP()
virNetDevOpenvswitchSetOnline()
[maybe some others]
2) modify src/network/bridge_driver.c:networkStartNetworkVirtual() to
check for a virtualport element with type openvswitch, and in that case
call the above virNetDevOpenvswitch*() functions in place of
virNetDevBridge*()
3) remove the restriction requiring forward mode == 'bridge' that you
mentioned in your original mail (since it will no longer be valid).
4) update src/docs/formatnetwork.html.in and
src/docs/schemas/network.rng to reflect that libvirt-managed virtual
networks can now use openvswitch for their bridges.
>
>
> When I omit the VLAN tags as follows I am able to create the network:
>
> <network>
> <name>ajith-net1</name>
> <virtualport type='openvswitch'/>
> <portgroup name='client_lan'>
> </portgroup>
> <portgroup name='client_wan'>
> </portgroup>
> </network>
>
> In my domain configs I specify the network like so:
>
> <interface type='network'>
> <source network='ajith-net1' portgroup='client_lan'/>
> <virtualport type='openvswitch'/>
(If the network has <virtualport type='openvswitch'>, it's redundant to
put it in the domain's <interface> definition.)
> <model type='virtio'/>
> </interface>
>
> The resulting ephemeral bridge(virbr1) looks like the following when my
> network(w/o vlans) and two domains are started. I don't know if the portgroup
> was meaningful, but it was accepted in the definition:
>
> $ sudo ovs-vsctl show
> <...>
> Bridge "virbr1"
> Port "vnet23"
> Interface "vnet23"
> Port "vnet25"
> Interface "vnet25"
> Port "virbr1"
> Interface "virbr1"
> type: internal
> Port "virbr1-nic"
> Interface "virbr1-nic"
> ovs_version: "1.9.3"
You apparently have openvswitch's "Linux host bridge compatibility"
package installed on your machine. If you didn't, the network definition
you have would have created a Linux host bridge rather than an
openvswitch bridge. libvirt doesn't contain any code that can create an
openvswitch bridge directly, so that's the only possible way this could
be happening. The problem is that when you use compatibility mode,
you're limited to the Linux bridge-utils API, which has no method of
specifying a vlan tag for individual ports (because Linux host bridges
lack that capability).
>
> My goal is assign unique vlan tags to the two portgroups. While the network
> and the domains are started, I can manually add tags to the ports:
>
> $ sudo ovs-vsctl set port vnet25 tag=2
> $ sudo ovs-vsctl set port vnet23 tag=1
> $ sudo ovs-vsctl show
> Bridge "virbr1"
> Port "vnet23"
> tag: 1
> Interface "vnet23"
> Port "vnet25"
> tag: 2
> Interface "vnet25"
> Port "virbr1"
> Interface "virbr1"
> type: internal
> Port "virbr1-nic"
> Interface "virbr1-nic"
> ovs_version: "1.9.3"
>
>
> This configuration behaves exactly as I want. I'm interested to learn how to
> prepare this without the manual step of tagging the ports outside of the
> libvirt operations.
>
> Ultimately my goal is to prepare isolated test environments that consist of
> several VM's attached to a similar qty of vlans. I intend to create many of
> these environments per host. I also recongnize that instead of portgroups, I
> could use separate networks altogether. From an administrative standpoint, I'd
> prefer to have one "network" per test environment, with several portgroups,
> instead of *many* networks.
Since this is all just numbers in memory (no real cables / switches),
there is little to no practical difference between having a single
bridge with lots of vlans, or having lots of bridges with no vlans. From
the point of view of client configuration, it would be the difference of:
[guest type 1]
<interface type='network'>
<source network='A' portgroup='1'/>
...
</interface>
[guest type 2]
<interface type='network'>
<source network='A' portgroup='2'/>
...
</interface>
versus
[guest type 1]
<interface type='network'>
<source network='A1'/>
...
</interface>
[guest type 2]
<interface type='network'>
<source network='A2'>
...
</interface>
and in the network config it would be:
<network>
<name>A</name>
....
<portgroup name='1'>
...
</portgroup>
<portgroup name='2'>
...
</portgroup>
</network>
versus:
<network>
<name>A1</name>
...
</network>
<network>
<name>A2</name>
...
</network>
One big difference is that you can do the latter today with existing
libvirt code (and you don't even need to have openvswitch installed on
your host). Unless you have > 255 guests on a single vlan, or need some
other openvswitch-specific feature not available with Linux host
bridges, I would just setup multiple networks and use the existing
libvirt networks.
> FWIW, I'm doing something similar on ESX, where I create a "vSwitch" and then
> create several "portgroups", each with its own vlan tag. These vSwitches are
> not attached to any physical interfaces. (My esx work is directly against the
> vsphere API, so I'm not sure if the same could be configured via libvirt) I'm
> looking to port this work to KVM. It seems like a pretty natural thing to
> support.
More information about the libvir-list
mailing list