[libvirt-users] Problem with net-define using Open vSwitch bridge

Tue Apr 9 16:42:14 UTC 2013

[Please do not top-post replies on a technical list. Put your responses
inline to make following the conversation easier.]

On 04/09/2013 11:56 AM, Will Dennis wrote:
> So, looks like a permissions problem.. From
> /var/log/libvirt/libvirtd.log:
>
> 2013-04-08 16:58:53.897+0000: 6743: info : libvirt version: 1.0.4
> 2013-04-08 16:58:53.897+0000: 6743: error : virCommandWait:2315 :
> internal error Child process (/usr/local/bin/ovs-vsctl --timeout=5 --
> --may-exist add-port vl20-ovsbr0 vnet0 -- set Interface vnet0
> 'external-ids:attached-mac="52:54:00:10:B0:EE"' -- set Interface vnet0
> 'external-ids:iface-id="6e4a326a-1595-e712-15ec-c4905725663e"' -- set
> Interface vnet0
> 'external-ids:vm-id="67e7842b-a47b-54df-bb7c-aca75368cd45"' -- set
> Interface vnet0 external-ids:iface-status=active) unexpected exit status
> 1: libvirt:  error : cannot execute binary /usr/local/bin/ovs-vsctl:
> Permission denied
> 2013-04-08 16:58:53.897+0000: 6743: error :
> virNetDevOpenvswitchAddPort:136 : Unable to add port vnet0 to OVS bridge
> vl20-ovsbr0: Operation not permitted
> 2013-04-08 16:58:53.937+0000: 6743: error : virCommandWait:2315 :
> internal error Child process (/usr/local/bin/ovs-vsctl --timeout=5 --
> --if-exists del-port) unexpected exit status 1: libvirt:  error : cannot
> execute binary /usr/local/bin/ovs-vsctl: Permission denied
>
> But ovs-vsctl does have read+exec set for 'others':
> # ls -la /usr/local/bin/ovs-vsctl
> -rwxr-xr-x 1 root root 3930719 Feb 26 17:12 /usr/local/bin/ovs-vsctl
>
> What else is needed?

libvirtd is running as root, so that should be plenty of permissions. I
assume you can run /usr/local/bin/ovs-vsctl from a root shell, correct?
Does your system have AppArmor or SELinux installed? Anything in their
logs that could indicate the reason for the denial? Anybody else have an
idea?

>
> The reason I want to use OVS "fake bridges" is to keep it simple for the
> team that will be using this virtualization server. If they just select
> the correct libvirt network (which is tied to the OVS "fake bridge")
> then the ports will be automatically tagged with the correct VLAN. (I
> think of OVS fake bridges as something akin to VMware "Port Groups" on
> an OVS level.)  I have defined libvirt networks as detailed in Kyle's
> blogpost you referenced, but there does not seem to be a way in the
> virt-manager UI to select the desired portgroup as defined in the
> network XML. (Yes, you can edit the VM's XML config and hack the
> portgroup into there, but that's not what I want these users to have to
> do...)

Sigh.

Yes, setting up a portgroup in the network definition for each vlan,
then telling the users to select the proper portgroup would be the right
way to do what you want, but unfortunately there has been little active
development to support new libvirt features in virt-manager for "awhile"
now, and the addition of portgroups to libvirt has happened in the meantime.

If you're familiar with python and would like to add support for that,
I'm sure virt-manager would *gladly* accept a patch. Otherwise,
hopefully there will be some new blood in virt-manager, and they'll
think this is a good request to take on.

At any rate, the generic nature of your error leads me to believe that
the problem isn't the "fake bridge" after all.

>
> Thanks again for your assistance!
> Will
>  
>
> -----Original Message-----
> From: sendmail [mailto:justsendmailnothingelse at gmail.com] On Behalf Of
> Laine Stump
> Sent: Tuesday, April 09, 2013 10:47 AM
> To: libvirt-users at redhat.com
> Cc: Will Dennis
> Subject: Re: [libvirt-users] Problem with net-define using Open vSwitch
> bridge
>
> On 04/08/2013 05:20 PM, Will Dennis wrote:
>> Thanks, Laine, for your reply. The OVS bridge named 'vl20-ovsbr0' does
>> exist, but it is an Open vSwitch "fake bridge" -- see 
>> http://blog.scottlowe.org/2012/10/19/vlans-with-open-vswitch-fake-brid
>> ge
>> s/ for why I'm doing this.
>> Here's some relevant output from my system:
>>
>> # ifconfig | grep encap
>> eth0      Link encap:Ethernet  HWaddr 00:25:90:aa:bb:cc
>> lo        Link encap:Local Loopback
>> ovsbr0    Link encap:Ethernet  HWaddr a2:c6:37:cd:90:4a
>> vl10-ovsbr0 Link encap:Ethernet  HWaddr b6:c4:8c:69:2b:06
>> vl20-ovsbr0 Link encap:Ethernet  HWaddr d6:d0:be:6e:b6:fb
>>
>> # ovs-vsctl --real list-br
>> ovsbr0
>>
>> # ovs-vsctl --fake list-br
>> vl10-ovsbr0
>> vl20-ovsbr0
>>
>> # ovs-vsctl show
>> 3e0d861b-efb7-46b1-af1b-4a76cd833558
>>     Bridge "ovsbr0"
>>         Controller "tcp:127.0.0.1:6633"
>>             is_connected: true
>>         Port "ovsbr0"
>>             Interface "ovsbr0"
>>                 type: internal
>>         Port "vl10-ovsbr0"
>>             tag: 10
>>             Interface "vl10-ovsbr0"
>>                 type: internal
>>         Port "vl20-ovsbr0"
>>             tag: 20
>>             Interface "vl20-ovsbr0"
>>                 type: internal
>>
>> # ovs-vsctl br-to-parent vl20-ovsbr0
>> ovsbr0
>>
>> virsh # net-dumpxml 20-net
>> <network>
>>   <name>20-net</name>
>>   <uuid>9e2c2ce7-a193-17ec-d6c0-3f115e759e29</uuid>
>>   <forward mode='bridge'/>
>>   <bridge name='vl20-ovsbr0' />
>>   <virtualport type='openvswitch'/>
>> </network>
> This looks fine.
>
>> (I have no XML from the guest, as virt-manager will not let me create 
>> it with the desired network...)
>>
>> Is the problem that libvirt only can define networks with "real" OVS 
>> bridges? That would be a bummer, since I need the VM's port tagged 
>> with a specific VLAN... Or is there another workable way to do this?
>>
> Since this is the first time I've heard of an ovs "fake bridge", I can't
> really say for sure if it works correctly. However, I now see from your
> 2nd message that the reason for your initial failure was that you were
> using a version of libvirt (0.9.4) that had no OVS support at all, and
> that you've since upgraded to libvirt 1.0.4.
>
> Meanwhile, the blog post that you reference above as the reason you're
> using a "fake bridge" is saying that the reason to do this is to
> overcome some unnamed bug in libvirt's OVS vlan support. That blog entry
> was written in October 2012, and mentions that "Kyle" (probably Kyle
> Mestery) had submitted a patch for [whatever the problem was]. I'm
> unable to determine which bug or which patch they're talking about from
> that information, but I don't believe there are any open bugs in OVS
> vlan support, so it must have been pushed and so would definitely be
> fixed in libvirt 1.0.4.
>
> So, my advice is 1) stop using "fake bridges" and 2) use libvirt's ovs
> vlan support, as described here:
>
>    http://www.siliconloons.com/?p=305
>
> (It would be useful to make these "fake bridges" work properly though.
> You can probably find the error message output by ovs-vsctl by looking
> in /var/log/messages or /var/log/libvirt/libvirtd.log (it will be logged
> by the libvirtd process). If you can post that message to this thread,
> maybe someone will understand why it's failing, and we can push a fix
> for it.)
>
>
> _______________________________________________
> libvirt-users mailing list
> libvirt-users at redhat.com
> https://www.redhat.com/mailman/listinfo/libvirt-users
>
>