[libvirt] Unable to provision VM attaching it directly to a OVS bridge

[Laine Stump]

On 10/29/2013 04:04 PM, Lucas Brasilino wrote:
> Moving this discussion from libvirt-users:
>
> Getting deeper and deeper :)
>
> $ egrep '^int vir*' util/virnetdevopenvswitch.c
> int virNetDevOpenvswitchAddPort(const char *brname, const char *ifname,
> int virNetDevOpenvswitchRemovePort(const char *brname
> ATTRIBUTE_UNUSED, const char *ifname)
> int virNetDevOpenvswitchGetMigrateData(char **migrate, const char *ifname)
> int virNetDevOpenvswitchSetMigrateData(char *migrate, const char *ifname)
>
> Libvirt by now basically support adding/removing port to a OVS bridge .
> Is there some planning on an OVS full support, like bridge creation/destroy,
> creating a VM attaching it to a OVS brige, etc, just like
> 'traditional' Linux bridge ?
> If so, Is anybody working on it ?
>
> regards
> Lucas Brasilino
> MSc Student @ Federal University of Pernambuco (UFPE)
> twitter: @lucas_brasilino
>
>
> 2013/10/29 Lucas Brasilino <lrbbs at cin.ufpe.br>:
>> Hi
>>
>> Getting deeper, the error is raised by 'virDomainCreateLinux()'.
>> Here comes the snippet of the XML argument passed to this call:
>>
>> (Pdb) print xmlDesc
>> <domain type='kvm'>
>>   <name>vm2</name>
>>   <uuid>3d713513-e8ee-994a-0eba-51128bd4b42e</uuid>
>> [...]
>>   <interface type='bridge'>
>>       <source bridge='databr0'/>
>>       <mac address='00:00:00:00:00:03'/>
>>       <model type='virtio'/>
>>     </interface>
>> [...]
>> </domain>
>>
>> So libvirt network driver is surely trying to use 'databr0' as the
>> common Linux bridge implementation, not an Open vSwitch bridge.
>>
>> I tried to add an
>>
>> <virtualport='openvswitch'/>
>>
>> element in the XML created by virt-install and use 'virsh create' to
>> create it but
>> seems that libvirt's network driver does not support Open vSwitch when
>> creating VM's.
>> Is there some planning to support it ?
>>
>> Thanks!
>>
>> regards
>> Lucas Brasilino
>> MSc Student @ Federal University of Pernambuco (UFPE)
>> twitter: @lucas_brasilino
>>
>> 2013/10/28 Lucas Brasilino <lrbbs at cin.ufpe.br>:
>>> Reposting from virt-tools mailing list:
>>>
>>> Hi!
>>>
>>> I'm facing a problem that could be triggered by some lacking
>>> of support from libvirt on Open vSwitch (or could be my mistake).
>>>
>>> I have interests in researching on virtual networks and
>>> SDN. To keep things simple, I've decided to use libvirt/virt-tools to
>>> manage VM's since my focus is on the network, instead of using a
>>> full feature system like OpenStack.
>>>
>>> I'm quite new with libvirt/virt-tools, but I have a good experience with
>>> openvswitch and other virtualizations technologies (which I dropped in
>>> libvirt/kvm favor).
>>>
>>> I'm using Fedora 19 packages
>>>
>>> openvswitch (1.11.0-1.fc19.x86_64)
>>> libvirt (1.0.5.6-3.fc19.x86_64)
>>> virt-install (0.10.0-4.fc19.noarch)
>>>
>>> I've created an OVS bridge (databr0) outside libvirt, and then defined it
>>> inside libvirt with:
>>>
>>> <network>
>>>   <name>databr0</name>
>>>   <forward mode='bridge'/>
>>>   <bridge name='databr0'/>
>>>   <virtualport type='openvswitch'/>
>>> </network>
>>>
>>> And then 'net-autostart' and 'net-start' it with virsh. Now I've got:
>>> #  virsh net-list
>>>  Name                 State      Autostart     Persistent
>>> ----------------------------------------------------------
>>>  databr0              active     yes           yes
>>>
>>> When I try to provision an VM, if I use the virt-install option
>>> "--network=bridge:databr0,model=virtio,mac=00:00:00:00:00:03"
>>> I got the following error:
>>>
>>> # virt-install --connect qemu:///system --virt-type kvm --name vm2
>>> --ram 768 --disk path=/home/lucas/local/vm/images/vm2.img --vnc
>>> --cdrom /home/lucas/local/vm/fc19-x86_64.iso
>>> --network=bridge:databr0,model=virtio,mac=00:00:00:00:00:03
>>> --os-type=linux --os-variant fedora19

AFAIK, virt-install has no method of specifying the <virtualport>
parameters for an interface. What you're asking for here is an interface
definition of:

     <interface type='bridge'>
       <source bridge='databr0'/>
       <model type='virtio'/>
       <mac address='00:00:00:00:00:03'/>
     </interface>

Since there is no <virtualport type='openvswitch'/>, this tells libvirt
to look for a standard Linux host bridge named databr0 and connect to
it. But you don't have a Linux host bridge named databr0, you have an
OVS bridge named databr0 (and you happen to also have a libvirt network
named databr0 which is defined to use an OVS bridge named databr0, but
you haven't referenced that network in your virt-install commandline.)

You need to change you virt-install commandline so that the --network
argument is something like this:

    --network network=databr0,model=virtio,mac=00:00:00:00:00:03

As long as the OVS service is running this should do what you want
(although you'll later probably want to add a "<virtualport
type='openvswitch'> element to the domain's <interface> element, so that
libvirt generates/stores a permanent interface-id for the domain rather
than generating a new one each time the domain is started).


>>>
>>> Starting install...
>>> ERROR    Unable to add bridge databr0 port vnet0: Operation not supported
>>> Domain installation does not appear to have been successful.
>>> If it was, you can restart your domain by running:
>>>   virsh --connect qemu:///system start vm2
>>> otherwise, please restart your installation.
>>>
>>> I just managed to create a VM when I use '--nonetworks' option and after
>>> I do a 'virtsh edit vm2' and add:
>>>
>>>   <interface type='bridge'>

If you're going to use <interface type='bridge'> then you don't need to
define the network. Alternately, if you are defining a network based on
the ovs bridge, you shoul define the interface like this:

     <interface type='network'>
       <source network='databr0'/>

(If you don't care about the interface-id (used internally by ovs), then
you don't even need the <virtualport> element in the <interface>
definition))


>>>       <mac address='00:00:00:00:00:03'/>
>>>       <source bridge='databr0'/>
>>>       <virtualport type='openvswitch'/>
>>>       <model type='virtio'/>
>>>       <address type='pci' domain='0x0000' bus='0x00' slot='0x05'
>>> function='0x0'/>
>>>     </interface>


You don't say what happens when you try to start this domain. I'm
guessing it won't give the same error.


>>>
>>> Well, I read elsewhere that openvswitch bridging isn't fully
>>> supported. Is it the case or I'm facing another kind of problem ?
>>>

It certainly is supported to the level that you're trying to use it.
Libvirt can define networks that use existing ovs switches, and connect
guest interfaces either via the aforementioned networks, or directly
using type='bridge'.

What libvirt doesn't support is any sort of management of the OVS
switches. In most cases that is better left up to a network management
application - there are just too many variables for libvirt to try and
support all of it. The one case that I have considered worth supporting
is to make libvirt's simple managed networks (for NAT, routed, and
isolated networks) support using Open vSwitch as the underlying
technology; for traditional use cases of a few QEMU guests the Linux
host bridge is just fine, but one case where it is inadequate is a large
host with > 255 guests that want to connect to the same network; this
number of guests is fairly easy to achieve when using LXC rather than
QEMU (because the memory requirements are significantly lower). I
haven't had the time to do this, but can explain the necessary code
additions to anyone who wants to take on the task.,