routing tables on two NICs for network monitoring

[Rick Stevens]

Shawn Iverson wrote:
> I have FC1 running nagios that monitors our network 24x7.  It has two NICs
> installed:  one to the internal network (eth0) and another to a cable modem
> via an ISP (eth1).  In case of a critical network failure after hours (i.e.
> core router), I want nagios to send pages/email via eth1 to alert since it
> will not be able to communicate information via the local network.
> 
> I am having trouble with the routing tables.  I have eth1 configured with
> DHCP to get IP/DNS information.  NIC eth0 is static.  I need gateways set on
> both so that nagios can monitor the internal network consisting of many
> subnets and send alerts on eth1.
> 
> I can only have one default gateway, so how do I set up another gateway?  I
> cheated and added another default gateway in rc.local for eth1, so the
> routing table has two default gateways:
> 
> Destination     Gateway         Genmask         Flags Metric Ref    Use
> Iface
> 172.16.1.0      *               255.255.255.0   U     0      0        0 eth1
> 10.10.0.0       *               255.255.0.0     U     0      0        0 eth0
> 169.254.0.0     *               255.255.0.0     U     0      0        0 eth1
> 127.0.0.0       *               255.0.0.0       U     0      0        0 lo
> default         172.16.1.1      0.0.0.0         UG    0      0        0 eth1
> default         10.10.0.254     0.0.0.0         UG    0      0        0 eth0
> 
> Well, all is well until one of the interfaces goes down or is unplugged.
> Also, I have no control as to which interface traffic traverses.
> 
> I did man page reading and found that perhaps I could let eth1 have the
> default gateway, set in ifcfg-eth1, and set up static network routes on eth0
> perhaps as follows:
> 
> route add -net 10.0.0.0 netmask 255.0.0.0 gw 10.32.0.254 dev eth0
> route add -net 192.168.0.0 netmask 255.255.0.0 gw 10.32.0.254 dev eth0
> 
> /etc/sysconfig/network-scripts/route-eth0 is my guess to add the above to
> make the changes permanent
> 
> GATEWAY0=10.32.0.254
> NETMASK0=255.0.0.0
> ADDRESS0=10.0.0.0
> GATEWAY1=10.32.0.254
> NETMASK1=255.255.0.0
> ADDRESS1=192.168.0.0

GOD NO!  Bad dog!  Bad!  NEVER assign a host an IP address that's the
same as the network (your "ADDRESS0=10.0.0.0" and
"ADDRESS1=192.168.0.0" lines).  Also NEVER assign a host the IP address
that is the broadcast address for the net (where the host bit is all
ones).

> Would this be the best way do accomplish what I need?  If my reasoning is
> correct, anything that is not bound for 10.x.x.x or 192.168.x.x will default
> to eth1, so nagios will effectively send alerts via eth1 while still being
> able to monitor all of our subnets via eth0.

Put the default gateway in /etc/sysconfig/network:

	GATEWAY=172.16.1.1

and remove it from any /etc/sysconfig/network-scripts/ifcfg-ethx file.
You should NEVER have more than one default route.  Each ifcfg-ethx file
should have the following lines at a minimum:

	DEVICE=
	BOOTPROTO=
	IPADDR=
	NETMASK=
	ONBOOT=

The "NETWORK=" and "BROADCAST=" are optional.

You shouldn't need any routes at all for your internal network, provided
the IP addresses you're hitting are all on the 10.0.0.0/8 network (in
other words, all the internal stuff is on 10.x.y.z).  By default, you've
set eth0 to be on that network.  If you need to speak to 192.168.0.0/16,
then add a static route.  The command would be:

	route add -net 192.168.0.0 netmask 255.255.0.0 device eth0

I'd highly recommend you do a bit of reading on how networks operate
before you start doing stuff like this--specifically on netmasks and
routing.  The config you've displayed will NOT work and may really hose
your network.
----------------------------------------------------------------------
- Rick Stevens, Senior Systems Engineer     rstevens at vitalstream.com -
- VitalStream, Inc.                       http://www.vitalstream.com -
-                                                                    -
-      Batteries not included.  Offer not valid in some states.      -
-           Your mileage may vary.  Void where prohibited.           -
----------------------------------------------------------------------