multiple network interfaces questions, RHEL 3 new install

Fri Jan 21 20:08:26 UTC 2005

Our new dual Opteron server (x86_64) running RHEL3, update 3 (Taroon)
has a K8s pro motherboard with on board networking.  It also has two
broadcom Gigabit Ethernet NICs.

The system sees eth0 as the motherboard 100bT, and eth1 adn eth2 as the
two Broadcom interfaces.  

Questions:
I want to use one GB interface to talk to the other cluster machines,
the other GB interface to be the samba server, and not mix traffic between
them.

I'd like to use the 100bT interface for mail only.

Where can I find info on setting things up this way (or is it impossible) ?
I know that I can bind samba to a particular interface, but how do I control
what interface talks to the local intranet and what interface only talks
to other cluster machines?

Will they need to be different sub-nets to make this work?

The output of netstat -r is here:
[root at cluster_master etc]# netstat -r
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
172.17.2.0      *               255.255.255.0   U         0 0          0 eth0
172.17.2.0      *               255.255.255.0   U         0 0          0 eth1
172.17.2.0      *               255.255.255.0   U         0 0          0 eth2
169.254.0.0     *               255.255.0.0     U         0 0          0 eth2
default         172.17.2.1      0.0.0.0         UG        0 0          0 eth0

I don't know what or where the 169.254.0.0 address is...

The output of ifconfig -a is here:
[root at cluster_master etc]# ifconfig -a
eth0      Link encap:Ethernet  HWaddr 00:E0:81:2C:74:33
          inet addr:172.17.2.29  Bcast:172.17.2.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:928092 errors:0 dropped:0 overruns:0 frame:0
          TX packets:218612 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1134994584 (1082.4 Mb)  TX bytes:40338127 (38.4 Mb)
          Interrupt:18 Base address:0xbc00 Memory:feafb000-feafb038

eth1      Link encap:Ethernet  HWaddr 00:E0:81:2C:73:86
          inet addr:172.17.2.30  Bcast:172.17.2.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:88001 errors:0 dropped:0 overruns:0 frame:0
          TX packets:70 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:6950013 (6.6 Mb)  TX bytes:4480 (4.3 Kb)
          Interrupt:24

eth2      Link encap:Ethernet  HWaddr 00:E0:81:2C:73:87
          inet addr:172.17.2.31  Bcast:172.17.2.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:85921 errors:0 dropped:0 overruns:0 frame:0
          TX packets:70 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:6791141 (6.4 Mb)  TX bytes:4480 (4.3 Kb)
          Interrupt:25

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:539071 errors:0 dropped:0 overruns:0 frame:0
          TX packets:539071 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:49468938 (47.1 Mb)  TX bytes:49468938 (47.1 Mb)

Here is the dmesg output for the network stuff:
Intel(R) PRO/100 Network Driver - version 2.3.43-k1
Copyright (c) 2004 Intel Corporation

divert: allocating divert_blk for eth0
e100: selftest OK.
e100: eth0: Intel(R) PRO/100 Network Connection
  Hardware receive checksums enabled
  cpu cycle saver enabled

tg3.c:v3.6RH (June 12, 2004)
divert: allocating divert_blk for eth1
eth1: Tigon3 [partno(BCM95704A7) rev 2003 PHY(5704)] (PCIX:100MHz:64-bit) 10/100/1000BaseT Ethernet 00:e0:81:2c:73:86
eth1: HostTXDS[1] RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[0]
divert: allocating divert_blk for eth2
eth2: Tigon3 [partno(BCM95704A7) rev 2003 PHY(5704)] (PCIX:100MHz:64-bit) 10/100/1000BaseT Ethernet 00:e0:81:2c:73:87
eth2: HostTXDS[1] RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[0]
divert: freeing divert_blk for eth0
divert: freeing divert_blk for eth1
divert: freeing divert_blk for eth2
ip_tables: (C) 2000-2002 Netfilter core team
Intel(R) PRO/100 Network Driver - version 2.3.43-k1
Copyright (c) 2004 Intel Corporation

divert: allocating divert_blk for eth0
e100: selftest OK.
e100: eth0: Intel(R) PRO/100 Network Connection
  Hardware receive checksums enabled
  cpu cycle saver enabled

ip_tables: (C) 2000-2002 Netfilter core team
e100: eth0 NIC Link is Up 100 Mbps Full duplex
tg3.c:v3.6RH (June 12, 2004)
divert: allocating divert_blk for eth1
eth1: Tigon3 [partno(BCM95704A7) rev 2003 PHY(5704)] (PCIX:100MHz:64-bit) 10/100/1000BaseT Ethernet 00:e0:81:2c:73:86
eth1: HostTXDS[1] RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[0]
divert: allocating divert_blk for eth2
eth2: Tigon3 [partno(BCM95704A7) rev 2003 PHY(5704)] (PCIX:100MHz:64-bit) 10/100/1000BaseT Ethernet 00:e0:81:2c:73:87
eth2: HostTXDS[1] RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[0]
ip_tables: (C) 2000-2002 Netfilter core team
tg3: eth1: Link is up at 100 Mbps, full duplex.
tg3: eth1: Flow control is on for TX and on for RX.
ip_tables: (C) 2000-2002 Netfilter core team
tg3: eth2: Link is up at 100 Mbps, full duplex.
tg3: eth2: Flow control is on for TX and on for RX.

-- 
ACCEL Services, Inc.| Specialists in Gravity, Magnetics |  1(713)993-0671 ph.
 2401 Fountain View |   and Integrated Interpretation   |  1(713)993-0608 fax
     Suite 320      |                                   |
 Houston, TX, 77057 |          Chuck Campbell           | campbell at accelinc.com
                    |  President & Senior Geoscientist  |

     "Integration means more than having all the maps at the same scale!"