[Libvir] Virtual network iptables rules

[Mark McLoughlin]

Hi Dan,

On Thu, 2007-04-05 at 02:44 +0100, Daniel P. Berrange wrote:
> Warning, this is a long & complicated email with lots of horrible details :-)
> 
> I've long been a little confused with the way iptables & bridging interacts,
> so set out to do some experiments. I added a -j LOG rule to every single chain
> in both the filter & nat tables, and then tried various traffic patterns, to
> see which chains were traversed & in which order. 

	Nice work ...

> Scenario 2: Virtual network
> ===========================
> 
>   net.bridge.bridge-nf-call-iptables = 1
> 
>   Host:  eth0 -> Internet
>          virbr0 -> MASQUERADE to eth0
> 
>   Guest: vif1.0 -> virbr0
> 
> 
> Traffic: Guest -> Google
> ------------------------
> 
> Out:
> 
> NAT-PREROUTING  IN=virbr0 OUT=       PHYSIN=vif1.0 SRC=192.168.122.47 DST=64.233.167.99
> FORWARD         IN=virbr0 OUT=eth0   PHYSIN=vif1.0 SRC=192.168.122.47 DST=64.233.167.99
> NAT-POSTROUTING IN=       OUT=eth0   PHYSIN=vif1.0 SRC=192.168.122.47 DST=64.233.167.99

	This really suprises me - I would have expected another one like:

FORWARD         IN=virbr0 OUT=virbr0 PHYSIN=vif1.0 PHYSOUT=virb0  SRC=192.168.122.47 DST=64.233.167.99

	Is it because the packets are coming in on bridge interface we don't
see any physdev matching? So, we would see it with Guest->Guest?

> For virtual networks there are basically 3 types of networking config we need to represent
> in terms of iptables rules, and these need to work for scenrios 1 & 2 - ie regardless of
> the magic sysctl knob.

	Well, IMHO, we should never be switching off the sysctl knob ourselves
- i.e. we shouldn't have it in xen/scripts/network-bridge - but I take
the point that a user might switch it off.

>   Problem: The INPUT rules are missing altogether for the isolated virtual network
>            so potentially DHCP/DNS will be blocked
>  Solution: Add them - simple bug.

	I fixed this in CVS, didn't I?

>   Problem: The POSTROUTING rule is too generic so it matches pretty much any kind
>            of traffic, from any virtual network, or even from VPN devices setup
>            by VPNC.
>  Solution: Only masquerade traffic whose source address is within the netmask
>            associated with the virtual network in question
> 
> 
>   Problem: The FORWARD rule is too generic, forwarding traffic to/from the
>            virtual network regardless of whether the dest/src IP address 
>            is within the netmask associated with the virtual network. Assuming
>            the first problem is setup to only masquerade valid IP addresses 
>            from the virtual network, this rule would then allow guests to
>            spoof their IP and have it forwarded off-host.
>  Solution: Only forward packets whose IP address is within the netmask 
>            associated with the virtual network
>  
> 
>   Problem: The policy of the FORWARD rule is ACCEPT, and/or later user defined
>            rules may inadvertently match on traffic from the virtual network,
>            again allowing through spoof traffic, or traffic from what should
>            be an isolated virtual network
>  Solution: There needs to be a catch-all REJECT rule associated with every
>            bridge device, in both directions
> 
> 
>   Problem: There is an extra physdev match per bridge device, and per guest
>            device. This is basically unneccessary since the previous rule
>            sets will already have allowed through the traffic. The physdev
>            matches also only work if net.bridge.bridge-nf-call-iptables = 1
>  Solution: Simply remove the per-device matches  
> 
> 
>   Problem: The POSTROUTING rule has a physdev match applied, which only works
>            if net.bridge.bridge-nf-call-iptables = 1.
>  Solution: Remove physdev match & masquerade based on network address associated
>            with the virtual network

	I guess the two main differences are 1) avoid physdev based rules
because they don't work with net.bridge.bridge-nf-call-iptables = 1 and
2) use network address based rules which I avoided because of pure
superstition and the feeling that IP based matching on a bridge was just
ugly.

	I haven't spent long thinking about these changes, but on the face of
it they all look well thought out and sensible. Definitely worth giving
it a shot.

Cheers,
Mark.