[libvirt-users] unable to ping from guests in virbr0 to guests in virbr1 network

[Marwan Tanager]

On Mon, Nov 19, 2012 at 12:09:53PM +0530, mallapadi niranjan wrote:
> Hi all,
> 
> I have 3 guests (2-RHEL4 and 1 RHEL6) and have some issues regarding
> networking between them. The 2 RHEL4 system's use default bridge virbr0 and
> get ip's of range 192.168.122.0/24  (192.168.122.207, 192.168.122.167)
> 
> I created another bridge (virbr1) with NAT forwarding (no dhcp). The
> network i choose was 192.168.100.0/24. And the third system (RHEL6) was
> assigned static ip-addres 192.168.100.101,
> 
> >>From the RHEL6 system which uses virbr1 is able to ping systems in
> 192.168.122.0/24 series , but  guest systems in 192.168.122.0/24 are not
> able to ping RHEL6 system (in virbr1) network.
> 
> >>From the RHEL4 guests i am able to ping the gateway ip's (192.168.122.1,
> 192.168.100.1) , but not the RHEL6 system
> 
> 
> Versions:
> Fedora release 16 (Verne)
> libvirt-0.9.6.3-1.fc16.x86_64
> qemu-kvm-0.15.1-8.fc16.x86_64
> 
> Any hints on what could be the problem

The problem is caused by the relative order of the iptables rules for those two 
networks.

When libvirt created virbr1 for the network 192.168.100.0/24 it inserted a 
couple of iptables rules on the FORWARDING chain for this interface, but it 
added them before the rules of virbr0 on the same chain. Those rules basically 
are ordered as follows (at least on my system which I suspect is different from 
yours since I also had the same problem):

1. Forward packets destined for the interface that are part of an established 
connection.
2. Forward packets coming from the interface.
3. Forward packets coming from and destined to the same interface (loopback).
4. Reject forwarding anything else to the interface.
5. Reject forwarding anything else from the interface.

Since those rules are inserted for every interface libvirt adds in this order, 
you can only ping from one interface to another if the set of rules for the 
interface you are pinging from comes before that of the one you're pinging to.  
Otherwise, rule number 4 of the destination interface will get in the way since 
iptables matches the packets against the rules in a chain according to their 
order on it.

A possible workaround:

iptables-save >/path/to/iptables/dump/file

then, edit the file to move all relevant rules that REJECT things on the 
FORWARDING chain to the end of the chain.

then put this command in your rc.local script:

cat /path/to/iptables/dump/file | iptables-restore

This solution is not reliable though because you will need to update the file 
containing the rules every time you add a new virtual network or update the 
iptables rules for some other reason.

Also, I hinted before at this problem on the list and someone pointed me that 
it's a bug, but nobody confirmed. Read my message on this topic with the 
subject: Inconsistent iptables forwarding rules for virtual networks, to get a 
clearer picture.

		
	Marwan