[libvirt-users] Problem with net-define using Open vSwitch bridge

Tue Apr 9 15:56:17 UTC 2013

So, looks like a permissions problem.. From
/var/log/libvirt/libvirtd.log:

2013-04-08 16:58:53.897+0000: 6743: info : libvirt version: 1.0.4
2013-04-08 16:58:53.897+0000: 6743: error : virCommandWait:2315 :
internal error Child process (/usr/local/bin/ovs-vsctl --timeout=5 --
--may-exist add-port vl20-ovsbr0 vnet0 -- set Interface vnet0
'external-ids:attached-mac="52:54:00:10:B0:EE"' -- set Interface vnet0
'external-ids:iface-id="6e4a326a-1595-e712-15ec-c4905725663e"' -- set
Interface vnet0
'external-ids:vm-id="67e7842b-a47b-54df-bb7c-aca75368cd45"' -- set
Interface vnet0 external-ids:iface-status=active) unexpected exit status
1: libvirt:  error : cannot execute binary /usr/local/bin/ovs-vsctl:
Permission denied
2013-04-08 16:58:53.897+0000: 6743: error :
virNetDevOpenvswitchAddPort:136 : Unable to add port vnet0 to OVS bridge
vl20-ovsbr0: Operation not permitted
2013-04-08 16:58:53.937+0000: 6743: error : virCommandWait:2315 :
internal error Child process (/usr/local/bin/ovs-vsctl --timeout=5 --
--if-exists del-port) unexpected exit status 1: libvirt:  error : cannot
execute binary /usr/local/bin/ovs-vsctl: Permission denied

But ovs-vsctl does have read+exec set for 'others':
# ls -la /usr/local/bin/ovs-vsctl
-rwxr-xr-x 1 root root 3930719 Feb 26 17:12 /usr/local/bin/ovs-vsctl

What else is needed?

The reason I want to use OVS "fake bridges" is to keep it simple for the
team that will be using this virtualization server. If they just select
the correct libvirt network (which is tied to the OVS "fake bridge")
then the ports will be automatically tagged with the correct VLAN. (I
think of OVS fake bridges as something akin to VMware "Port Groups" on
an OVS level.)  I have defined libvirt networks as detailed in Kyle's
blogpost you referenced, but there does not seem to be a way in the
virt-manager UI to select the desired portgroup as defined in the
network XML. (Yes, you can edit the VM's XML config and hack the
portgroup into there, but that's not what I want these users to have to
do...)

Thanks again for your assistance!
Will

-----Original Message-----
From: sendmail [mailto:justsendmailnothingelse at gmail.com] On Behalf Of
Laine Stump
Sent: Tuesday, April 09, 2013 10:47 AM
To: libvirt-users at redhat.com
Cc: Will Dennis
Subject: Re: [libvirt-users] Problem with net-define using Open vSwitch
bridge

On 04/08/2013 05:20 PM, Will Dennis wrote:
> Thanks, Laine, for your reply. The OVS bridge named 'vl20-ovsbr0' does

> exist, but it is an Open vSwitch "fake bridge" -- see 
> http://blog.scottlowe.org/2012/10/19/vlans-with-open-vswitch-fake-brid
> ge
> s/ for why I'm doing this.
> Here's some relevant output from my system:
>
> # ifconfig | grep encap
> eth0      Link encap:Ethernet  HWaddr 00:25:90:aa:bb:cc
> lo        Link encap:Local Loopback
> ovsbr0    Link encap:Ethernet  HWaddr a2:c6:37:cd:90:4a
> vl10-ovsbr0 Link encap:Ethernet  HWaddr b6:c4:8c:69:2b:06
> vl20-ovsbr0 Link encap:Ethernet  HWaddr d6:d0:be:6e:b6:fb
>
> # ovs-vsctl --real list-br
> ovsbr0
>
> # ovs-vsctl --fake list-br
> vl10-ovsbr0
> vl20-ovsbr0
>
> # ovs-vsctl show
> 3e0d861b-efb7-46b1-af1b-4a76cd833558
>     Bridge "ovsbr0"
>         Controller "tcp:127.0.0.1:6633"
>             is_connected: true
>         Port "ovsbr0"
>             Interface "ovsbr0"
>                 type: internal
>         Port "vl10-ovsbr0"
>             tag: 10
>             Interface "vl10-ovsbr0"
>                 type: internal
>         Port "vl20-ovsbr0"
>             tag: 20
>             Interface "vl20-ovsbr0"
>                 type: internal
>
> # ovs-vsctl br-to-parent vl20-ovsbr0
> ovsbr0
>
> virsh # net-dumpxml 20-net
> <network>
>   <name>20-net</name>
>   <uuid>9e2c2ce7-a193-17ec-d6c0-3f115e759e29</uuid>
>   <forward mode='bridge'/>
>   <bridge name='vl20-ovsbr0' />
>   <virtualport type='openvswitch'/>
> </network>

This looks fine.

>
> (I have no XML from the guest, as virt-manager will not let me create 
> it with the desired network...)
>
> Is the problem that libvirt only can define networks with "real" OVS 
> bridges? That would be a bummer, since I need the VM's port tagged 
> with a specific VLAN... Or is there another workable way to do this?
>

Since this is the first time I've heard of an ovs "fake bridge", I can't
really say for sure if it works correctly. However, I now see from your
2nd message that the reason for your initial failure was that you were
using a version of libvirt (0.9.4) that had no OVS support at all, and
that you've since upgraded to libvirt 1.0.4.

Meanwhile, the blog post that you reference above as the reason you're
using a "fake bridge" is saying that the reason to do this is to
overcome some unnamed bug in libvirt's OVS vlan support. That blog entry
was written in October 2012, and mentions that "Kyle" (probably Kyle
Mestery) had submitted a patch for [whatever the problem was]. I'm
unable to determine which bug or which patch they're talking about from
that information, but I don't believe there are any open bugs in OVS
vlan support, so it must have been pushed and so would definitely be
fixed in libvirt 1.0.4.

So, my advice is 1) stop using "fake bridges" and 2) use libvirt's ovs
vlan support, as described here:

   http://www.siliconloons.com/?p=305

(It would be useful to make these "fake bridges" work properly though.
You can probably find the error message output by ovs-vsctl by looking
in /var/log/messages or /var/log/libvirt/libvirtd.log (it will be logged
by the libvirtd process). If you can post that message to this thread,
maybe someone will understand why it's failing, and we can push a fix
for it.)