Duplicate Echo Replies with Channel Bonding

[Vincent Aniello]

I am running bonding with connections to two switches in an
active/backup configuration (mode=1) with eth0 the  active interface and
eth1 the backup.  In this mode both interfaces receive packets, but only
the active interface transmits packets.  I am running RedHat Enterprise
Linux 3.0 (kernel version 2.4.21-27.0.2.Elsmp).

When both eth0 and eth1 are up and I ping from Host C to Host A I get
duplicate ICMP echo replies.  I believe this is also causing duplicate
traffic with other types of traffic as well.

I believe I know why this is happening and I would like to see if anyone
else has any input on this and hopefully  a solution that does not
result in losing the redundancy bonding provides.

My network topology is as follows:

                      |          |           |          |
                      | Firewall |           | Firewall |
                      +-----+----+           +-----+----+
                            |                      |
                            |                      |
 +----------+         +-----+----+           +-----+----+
+----------+
 |          |   T1    | Router A |  Ethernet | Router B |    T1    |
|
 | Router D +---------+          |-----------|          +----------+
Router C |
 |          |         |  Active  |           | Standby  |          |
|
 |          |         |  Gateway |           | Gateway  |          |
|
 +----+-----+         +-----+----+           +-----+----+
+-----+----+
      |              192.168.1.3/24         192.168.1.4/24
|
      |                     | <-HSRP 192.168.1.1-> |
|
 +----+-----+         +-----+----+           +-----+----+
+-----+----+
 |          |         |          |           |          |          |
|
 | Switch D |         | Switch A +-----------+ Switch B |          |
Switch C |
 |          |         |          +-----------+          |          |
|
 |          |         |          |           |          |          |
|
 +----+-----+         +-----+----+           +-----+----+
+-----+----+
      |eth0                 |                      |
|eth0
  +--------+                |      +--------+      |
+--------+
  | Host D |                +------+ Host A +------+                |
Host C |
  +--------+                  eth0 +--------+ eth1
+--------+
                                     bond0                         
                                192.168.1.20/24


According to my theory the path the echo request packets are taking is
as follows:

  Host c -> Switch C -> Router C -> Router B -> Switch B
-+-------------> Host A eth1
                                                          |
                                                          +-> Switch A
-> Host A eth0

The destination network 192.168.120.0/24 exists on both Router A and
Router B and HSRP is used for failover  between them, with Router A
being the primary gateway.

The destination network is local to Router B and When the echo request
gets to Router B this router has the MAC address for 192.168.1.20 in its
ARP cache, and if it doesn't it requests it.  

Router B sends the Ethernet frame to the MAC address for 192.168.1.20.
The interfaces eth0, eth1, and bond0 have  the same MAC address on the
server.  Switch B does not have the MAC address in its MAC address table
and floods  the ports.  In flooding the ports the server receives the
packet on eth1.  The packet is also sent through the  connection between
Switch B and Switch A and received on eth0 of the server.  

Since the packet is received twice by the server (on eth0 & eth1) two
echo replies are sent along the following  path:

   Host A eth0 -> Switch A -> Router A -> Router B -> Router C -> Switch
C -> Host C

The reply packet is sent to the primary gateway IP address on Router A.
Switch A learns the MAC address of the  server, but Switch B does not.  

The switches only learn a MAC address when traffic is received from a
host on the switch port.  The interface eth0  is the active bonding
interface all outbound traffic is sent from this interface and therefore
the MAC address is  only learned on the switch port that eth0 is
connected to.  Switch B never learns the MAC address for the server  and
therefore the duplicate packets never stop when pining Host A from Host
C.

If I ping Host A from Host D I do not receive duplicate packets.  The
echo requests are only received once  (through eth0) on the server.  The
path the packets travel in this setup is:

  Echo Request: Host D -> Switch D -> Router A -> Switch A -> Host A
eth1

  Echo Reply: Host A eth0 -> Switch A -> Router A -> Router D -> Switch
D -> Host D

If I shut down one of the bonded (eth0 or eth1) interfaces I stop
receiving duplicate packets.

Has anyone else experienced this problem and know how to fix it?

Thanks.

--Vincent