[libvirt] VLAN-based direct interface

[Ed Swierk]

In my setup, the network interfaces of several KVM-based VMs all
communicate with the outside world via a single host interface, which
is in turn connected to an external switch that handles all network
policy to/from/between VMs. Traffic between the host and the switch is
segregated by VLAN ID, one per vNIC.

When I spin up a new VM with one vNIC assigned (say) VLAN ID 101, I
run ip link add link eth0 name eth0.101 type vlan id 101. In the
libvirt config I write

<interface type='direct' trustGuestRxFilters='yes'>
  <source dev='eth0.101' mode='passthrough'/>
  <model type='virtio'/>
</interface>

This causes libvirt to create a macvtap interface attached to eth0.101
in passthrough mode.

So far, so good. The guest OS thinks it has a plain old NIC on an
untagged switch port. When the guest OS joins an IPv6 link-local
solicited-node multicast group, QEMU tells libvirt to add the
corresponding multicast MAC address to the macvtap interface. I can
even run a trunk port all the way through to the VM, with traffic
double-tagged through the switch-host hop.

Trouble arises if I change the model from virtio to e1000, to support
an application that doesn't (yet) have virtio drivers. QEMU doesn't
expose state via query-rx-filter for e1000 interfaces, so libvirt
can't sync multicast MAC addresses from the guest to the macvtap
interface, so the guest can't talk IPv6.

Ideally QEMU would support query-rx-filter for e1000 and other NIC
models. But for my application all the careful syncing of guest rx
filter state is overkill: I need the macvtap interface to pass all
traffic between the guest and host with no filtering at all. I can
achieve this simply by enabling promiscuous mode on the macvtap
interface. But libvirt creates the interface, and I don't see any way
to get libvirt to enable promiscuous mode and leave it enabled.

It wouldn't be hard to add another configuration flag that tells
libvirt to just put the macvtap interface into promiscuous mode and
skip syncing guest rx filters. Perhaps add another value for
trustGuestRxFilters, like 'promisc'?

However just solving that minor problem seems like piling one hack on
top of another: taking a normal interface and attaching a VLAN
interface to it, then attaching a macvtap interface to that, and
beating the mac out of the macvtap until it behaves like a dumb bridge
(or, attaching a bridge to the VLAN interface, creating a tap
interface, and enslaving the tap to the bridge, old-school).

What I'm really after is just another kind of direct interface that
attaches to a host interface, but instead of directing traffic to the
guest by MAC address it uses the VLAN ID (and also adds/strips the
tag). libvirt could implement this by automating all the steps I just
described. Or if the kernel ever sprouts the VLAN equivalent of the
macvtap driver (vlanvtap?), libvirt could just use that.

Any opinions on first-class support for a VLAN-based direct interface,
versus just allowing configuration of promiscuous mode on a macvtap
direct interface?

--Ed