[libvirt] 802.1q vlan-tagging support for libvirt

[Daniel P. Berrange]

On Tue, Apr 07, 2009 at 06:32:43PM -0700, David Lutterkort wrote:
> On Tue, 2009-04-07 at 18:39 -0300, Klaus Heinrich Kiwi wrote:
> > I was thinking about the semantics I described above. It ultimately
> > means that we'll have a bridge for each VLAN tag that crosses the trunk
> > interface. So for example if guests A, B and C are all associated with
> > VLAN ID 20, then:
> > 
> > eth0 -> eth0.20 -> br0 -> [tap0, tap1, tap2]
> > 
> > (where tap[0-3] are associated with guests A, B, C respectively)
> 
> Yes, I think that's how it should work; it would also mean that you'd
> first set up eth0 as a separate interface, and new bridge/vlan interface
> combos afterwards. AFAIK, for the bridge, only bootproto=none would make
> sense.
> 
> > The things that concerns me the most are:
> >  1) How scalable this really is
> 
> I don't know either ... we'll find out ;)

I don't think that's really a scalability problem from libvirt's POV. I
know people use this setup quite widely already even with plain ifcfg-XXX
scripts. Any scalability problems nmost likely fall into the kernel /
networking code and whether it is good at avoiding unneccessary data
copies when you have stacked  NIC -> VLAN -> BRIDGE -> TAP

> 
> >  2) The semantics are really different from how physical, 802.1q-enabled
> > switches would work.
> > 
> > Because (2) really creates new switches for each new VLAN tag, I wonder
> > how management would be different from what we have today with physical
> > switches (i.e., defining a port with a VLAN ID, assigning that port to a
> > physical machine) - unless we hide it behind libvirt somehow.

I think one thing to consider is the difference between the physical and
logical models.  The libvirt API / representation here is fairly low 
level, dealing in individuals NICs. I think management apps would likely
want to present this in a slightly alternate way dealing more in logical
entities than physical NICs. eg oVirt's network model is closer to the
one you describe where the user defines a new switch for each VLAN tag.
It then maps this into the low level physical model of individual NICs
as needed. I think it si important that libvirt use the physical model
here  to give apps flexibility in how they expose it to users.

> 
> The reason we are creating all those bridges isn't the VLAN's - it's
> that we want to share the same physical interface amongst several
> guests. And I don't know of another way to do that.
> 
> > Are there other options? Since a tagged interface like eth0.20 is kind
> > of a virtual interface itself, would it be appropriate to use those
> > directly?
> 
> You can use it directly, I just don't know how else you would share it
> amongst VM's without a bridge.

In the (nearish) future NICs will start appearing with SR-IOV capabilities.
This gives you one physical PCI device, whcih exposes multiple functions.
So a single physical NIC appears as 8 NICs to the OS. You can thus directly
assign each of these virtual NICs to a different VM directly,a voiding the
need to bridge them.

I don't think its worth spending too much time trying to come up with other
non-bridged NIC sharing setups when hardware is about todo it all for us :-)

Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|