[libvirt] [RFC Incomplete Patch] Libvirt + Openvswitch

[kmestery]

On Jan 30, 2012, at 2:41 PM, Dan Wendlandt wrote:
> Hi Kyle!  Funny how we keep bumping into each other...  I hope you're
> keeping warm in Minnesota :)
> 
> On Fri, Jan 27, 2012 at 11:22 AM, kmestery <kmestery at cisco.com> wrote:
>> Hi Dan:
>> 
>> We at Cisco have been looking at this as well, and in fact were going to propose some things in this area as well. Our proposal (which frankly just builds on top of what you have):
> 
> I agree, I think the two proposals are complementary.  Our first goal
> was to enable the basic mode of plugging an interface into an OVS
> bridge that was created outside of libvirt.  This would require
> changes to the <interface> XML only, and would mirror how libvirt
> already let's one plug into an existing bridge using <interface
> type="bridge">.
> 
This makes sense.

> The second step would be to also allow libvirt to actually create +
> configure the OVS bridges as well.  This I believe would map very
> closely to the XML you and Roopa have suggested.  We would need to put
> some thought into what of the many configuration options on an OVS
> bridge need to be exposed as part of the OVS <network> XML (e.g., how
> to configure an OpenFlow controller, etc.).  These are definitely
> discussions worth having, but I was hoping to start out with a fairly
> clean initial patch to achieve our first goal.
> 
OK, this makes sense too.

> I do like the idea of using the virtual port construct even in the
> initial <interface> only case.  For example:
> 
> <interface type='bridge'>
>  <bridge name='br0'>
>  <virtualport type="openvswitch">
>    <parameters interfaceid='xyzzy'/>
>  </virtualport>
> </interface>
> 
> This would seem to create a nice parallel with the model you proposed
> where <virtualport> is used with <interface type="network">.   I'm
> still not sure whether the "type=openvswitch" should be an attribute
> of the <interface>, <bridge>, or <virtualport> element.  Either way, I
> think the underlying code will be treating OVS + Linux Bridge as two
> different cases, so I believe this is really just a matter of
> consistently of presentation in XML.
> 
I think fundamentally an Open vSwitch bridge is different from a standard Linux, thus the "type=openvswitch" needs to be a part of the <bridge> for sure. I like adding it to the <virtualport> element above.

I'm reworking the patch you sent last week to tie these all together, let me know if you plan to resubmit your patch as well so we can work together and not duplicate effort.

Thanks!
Kyle

> Dan
> 
> 
>> 
>> The proposal at hand is to add some libvirt configuration as follows:
>> 
>> <network>
>>   <name>ovs-network</name>
>>   <forward mode='ovs' dev='br0'>
>>   <script path='/etc/qemu-ifup-ovs'/>
>>   </forward>
>> </network>
>> 
>> This would setup the OVS network entity, and sets up a forwarding mode of "ovs", which indicates Open vSwitch is used to forward traffic.
>> 
>> <interface type='network'>
>>   <source network='ovs-network'/>
>>   <virtualport type='openvswitch'>
>>       <parameters interfaceid="interface-xyz"/>
>>   </virtualport>
>> </interface>
>> 
>> This model of exposing parameters to virtualport types allows for further expansion as new interface types and parameters are added.
>> 
>> Thoughts?
>> 
>> Thanks,
>> Kyle
>> 
>> On Jan 27, 2012, at 4:58 AM, Dan Wendlandt wrote:
>>> 
>>> Hello all,
>>> 
>>> I know of many people who want to spin up VMs using libvirt + kvm/qemu and attach those VMs to an openvswitch bridge (see: http://www.openvswitch.org).   However, the only way I know of to get this working is a kludge that uses to tap devices along with <interface type="ethernet"> while running ovs-vsctl outside of libvirt.  Even worse, doing this on RHEL/Fedora seems to require privilege tweaks (e.g., running qemu as root, not dropping capabilities), which may not be acceptable for production deployments (see:  http://fedoraproject.org/wiki/How_to_debug_Virtualization_problems#Errors_using_.3Cinterface_type.3D.27ethernet.27.2F.3E ).
>>> 
>>> So I would like to start taking steps toward better libvirt/openvswitch integration.  My initial step has the fairly limit goal of enabling kvm/qemu VM NICs to attach to an openvswitch bridge in much the same way VM NIC can already attached to the linux bridge.  For example, specifying:
>>> 
>>> <interface type="openvswitch">
>>> <source bridge="br0"/>
>>> <mac address="ca:fe:de;ad:be:ef"/>
>>> </interface>
>>> 
>>> I also wanted to add a new child element of <interface> that can be used to specify some OVS specific configuration.  To start, I just want to expose a single 'interfaceid' value (this parameter is specific to openvswitch).  Extending the previous example:
>>> 
>>> <interface type="openvswitch">
>>> <source bridge="br0"/>
>>> <mac address="ca:fe:de;ad:be:ef"/>
>>> <openvswitchport interfaceid="interface-xyz"/>
>>> </interface>
>>> 
>>> For this first step (see attached patch), I am only targeting the model where the OVS bridge has been created externally ahead of time.  I am not tackling any of the "network" logic that actually creates/destroys bridges, as it is not needed for my target use case.
>>> 
>>> A couple notes about the attached patch:
>>> - I haven't actually run this code.  I was just poking around the libvirt code to learn about it and figured that a diff was the most concrete way to propose what I was thinking of doing.  I would be curious for pointers about big chunks of work that I may have missed (for example, I haven't added any tests yet).
>>> - the code was modeled on the network interface "bandwidth" code, that calls out to 'tc' to configure bandwidth rates.  Ideally we would be able to make direct C calls to OVS (and we plan to make that possible in the future), but calling an external utility right now is the only viable path.
>>> - no attention was paid to style guidelines.  Will do that before any real submission.
>>> - I wasn't very clear on the notion of an "actual" net def, as opposed to a normal net def.  What's the best place to look for documentation on this?
>>> 
>>> Anyway, I'm primarily looking for feedback about whether I'm barking up the right tree before I spend time debugging or polishing the patch, so I would appreciate thoughts, advice, etc.  Thanks,
>>> 
>>> Dan