[libvirt-users] Issue with macvtap bridge and forwarding

[Matt LaPlante]

I have three hosts running Ubuntu 12.04 (libvirt 0.9.8).  The
configuration is one host running on bare metal while the other two
are KVM guests.

The first guest is my network router.  It has a direct connection to a
physical nic going out to the internet, and a bridged connection to a
nic for the lan.  The host has ip forwarding enabled and forwards my
lan traffic back and forth to the internet.

The second host is a "lan" machine, which is also on the bridged lan
nic.  There are several other physical hosts also on the switched lan
network this nic connects to.

Both hosts are configured on the lan tap as follows (different mac):

<interface type='direct'>
  <mac address='13:54:21:1f:f3:42'/>
  <source dev='eth1' mode='bridge'/>
  <model type='virtio'/>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>

Everything on the firewall host seems to work OK.  Hosts elsewhere on
the LAN can connect to it, and they have no problem routing through it
and out to the internet and back.  That is, traffic passes through
both nics and this host successfully.

The other guest can also be reached successfully from machines on the
LAN.  I can ping it and I can ssh to it.  I also note that it seems to
be able to be able to talk to the "router" host over the vtap bridge:
it can perform dns lookups against the router host, and they can seem
to reach each other's ports.

Where I'm getting stuck is that for whatever reason, the second guest
apparently cannot reach the internet via my router host.  It's the
only host anywhere on the lan that apparently can't pass forwarding
traffic via the router guest, and the only common feature appears to
be the macvtap bridge.  This issue *does not* happen when using a
common linux bridge in the otherwise same configuration and the same
hosts.  In the linux bridge scenario, the lan guest forwards traffic
via the router guest fine.  But when switching to the macvtap
configuration, suddenly the lan guest no longer forwards traffic via
the router guest and out to the internet.

What appears to be happening is that the traffic is crossing the
bridge, but the router host does not classify it or masq it properly,
and it never makes the internet trip as expected.  Further extending
my suspicions, initiating an outbound http connection does not raise
an entry in the conntrack table on the router for the problem host.

So my question is, what is it about macvtap bridge that would cause
traffic coming over via the tap bridge to be routed differently than
traffic coming *up* the bridge from the physical interface, or via a
traditional linux bridge?