FC1 multiple ethernet interface issue

[Bob Chiodini]

On Tue, 2005-02-15 at 09:02 -0500, Scot L. Harris wrote:
> On Tue, 2005-02-15 at 08:35, David Benigni wrote:
> > Here is the dump from ip route list for various modes of the nics.
> > 
> 
> (personally I prefer the netstat -rn output, I guess that makes me old
> school.  :)  )
> 
> > eth0 up and eth1 down (normal boot):
> > 10.1.0.0/16 dev eth0  scope link 
> > 127.0.0.0/8 dev lo  scope link 
> > default via 10.x.y.1 dev eth0 
> > 
> 
> That looks fine, single interface eth0 used.
> 
> > eth0 up and eth1 up (via ifup eth1):
> > 10.1.0.0/16 dev eth1  scope link 
> > 10.1.0.0/16 dev eth1  proto kernel  scope link  src 10.x.y.121 
> > 127.0.0.0/8 dev lo  scope link 
> > default via 10.x.y.1 dev eth1 
> > 
> 
> Note that the default route has switched to the eth1 interface. 
> Actually it looks like all the routes are using eth1.  What happened to
> eth0?  It really looks like you would have zero traffic going out over
> the eth0 interface in this configuration.  You should be able to look at
> the ifconfig information to get the stats in this setup.  You may see
> received data but I doubt you would see much if any transmit data on
> eth0.   
> 
> > eth0 up and eth1 down (via ifdown eth1):
> > 127.0.0.0/8 dev lo  scope link 
> > 
> 
> Would expect this result since everything was routing out the eth1
> interface.  So this result makes sense and appears to be correct.
> 
> > eth0 up and eth1 down(via /etc/init.d/network restart):
> > 10.1.0.0/16 dev eth0  scope link 
> > 127.0.0.0/8 dev lo  scope link 
> > default via 10.x.y.1 dev eth0
> > 
> 
> Back to square one.
> 
> At this point the problem is not at the point you shutdown eth1.  It is
> when you bring eth1 up and all your routes get pushed out that
> interface.  It is that point that the system must be looking at the
> addressing you are using and says the new interface has the same subnet
> as the old one so all traffic can go this way.  Depending on the actual
> IP address on the interface it may decide that based on the highest or
> lowest IP address.  Not sure about that but that appears to be what is
> happening.
> 
> So if you think this has been working with the two interfaces splitting
> traffic, it probably has not.  Look at the packet stats ifconfig gives
> you for each interface.  Setup some file transfers or something to each
> IP address for testing.  I suspect you will see all outbound traffic on
> eth1 and nothing on eth0, even for local LAN traffic.
> 
> 
> -- 
> Scot L. Harris
> webid at cfl.rr.com
> 

I suspect that the moving of the default route to eth1 and the removing
of the default route when eth1 is taken down has to do with the lookup
of the subnet associated with the router and the interface.  The default
route disappears because the interface with the router's subnet goes
down.  The matching algorithm does not take into account that there is
another interface on the same subnet.  It might be a bug, but most
systems are not configured with two interfaces in the same subnet
without some sort of bonding or bridging.

I've tried to sort out Dave's posting (fix your email client to not
create a new email thread for each response) and conclude that there is
one particular host he wants to communicate with on one of the
interfaces and everyone else on the other.  Why, and is this host
directly connected?  It might be easier to burn a private subnet for
this one host and let linux be it's default gateway (NAT and ip
forwarding).

Otherwise, you might be able to do some tricks with iptables, but could
be messy.

If I misread the posting, then in the words of Rosanne Rosanna Dana,
never mind.

Bob...