[libvirt-users] Strange connectivity issues with bridged networking and masquerade

[Kolja Scheffler]

Hi all,

I'm currently in the process of building a 2-Node libvirt/KVM Cluster
and ran into some issues regarding the network connectivity of our
virtual machines.

Our setup seemed to work fine, we were able to browse to Google and our
own company website and some others from within the VM. Then we tried
microsoft.com to download some Windows iso images from MS Partner
Network. The page started to load, but only a few elements became
visible - then it sticked to: "Loading data from
microsoft.com" ...forever. A few other examples that do not work are:

* www.opera.com
* www.amazon.com
* www.speedtest.net

All of these pages load without any problem, when I access them from my
laptop or even with Firefox via X-Forwarding launched directly on the
hypervisor system. From within the VMs they just refuse to finish
loading. The only thing those pages have in common, as far as I can see,
is that they heavily utilize CDNs like Amazon Cloudfront or Akamai.

The idea behind our setup is, that all virtual machines communicate on
the 192.168.3.0/24 network. The nodes have a VLAN connection on eth1. To
allow connections between VMs on different hosts, we created the bridge
device br1 with eth1 attached and added the VMs to it. eth0 provides
internet access with xx.xx.220.0 as additional public failover ip.

We added 192.168.3.254 as additional IP to one of the node's br1 device
to use it as the default gateway for the VMs. This IP can be migrated
between the nodes.

Our setup looks like this:
                  ____________
                 /            \
                (   Internet   )
                 \____________/
                   /        \
Node1:             |        |     Node2:        
                   |        |
xx.xx.217.8     (eth0)    (eth0)    xx.xx.217.10
xx.xx.220.0        \
                 {Masq.}
                   /
                (eth1)----(eth1)
                   |        |
192.168.3.1     [br1 ]    [br1 ]     192.168.3.2
192.168.3.254      |        |              
                   |        |
192.168.3.50   (vnet0)    (vnet0)   192.168.3.75

----------
  iptables looks like this:
  root at vm01:~# iptables -S
  -P INPUT ACCEPT
  -P FORWARD ACCEPT
  -P OUTPUT ACCEPT
  -A FORWARD -d 192.168.3.0/24 -o br1 -m state --state
RELATED,ESTABLISHED
  -j ACCEPT
  -A FORWARD -s 192.168.3.0/24 -i br1 -j ACCEPT
  -A FORWARD -i br1 -o br1 -j ACCEPT
  -A FORWARD -i eth0 -o eth0 -j ACCEPT
  root at vm01:~# iptables -S -t nat
  -P PREROUTING ACCEPT
  -P INPUT ACCEPT
  -P OUTPUT ACCEPT
  -P POSTROUTING ACCEPT
  -A POSTROUTING -s 192.168.3.0/24 ! -d 192.168.3.0/24 -j MASQUERADE
  -A POSTROUTING ! -s 192.168.3.0/24 -d 192.168.3.0/24 -j MASQUERADE
---------

Some additional information that might be helpful:
  root at vm01:~# virsh version
  Compiled against library: libvir 0.9.12
  Using library: libvir 0.9.12
  Using API: QEMU 0.9.12
  Running hypervisor: QEMU 1.1.2
---------
  root at vm01:~# uname -a
  Linux vm01.cluster 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1 x86_64
GNU/Linux
---------

I don't know if it's really libvirt-related but perhaps someone here has
an idea what to try. Any advice on this is really appreciated, as  I am
at my wits' end. Thank you in advance... :)

Kind regards
Kolja Scheffler
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: This is a digitally signed message part
URL: <http://listman.redhat.com/archives/libvirt-users/attachments/20130823/147d22ef/attachment.sig>