[K12OSN] Network unreachable.

[Les Mikesell]

James P. Kinney III wrote:
> This is beginning to sound like a module loading issue. Because you are
> swapping out cards presumably with different drivers and certainly with
> different MAC addresses, each change will require updating the
> configuration of the devices.  I am unclear on what distro you are
> running. 
> 
> Second note. Modules get loaded in the order they are found in a file
> called /etc/modules.conf (for current systems. Older systems
> used /etc/conf.modules). However many motherboards try to power on
> devices based on bus ID. That is typically done low number to high based
> on slot location physically relative to the CPU location. So the top pci
> slot is 0, the next one down is 1. But some boards do this in reverse
> (high to low). So the removal of the second nic causes the first nic to
> function properly which suggests the configuration of the second is out
> of order, has the wrong MAC on both, or all of the above. The mii-tool
> mentioned earlier will give you data on the interface specifics as well
> as providing a means to adjust hardware settings (like full-duplex,
> etc).

In recent 2.6 kernels (Centos 5.x, probably FC6), at least some of the 
device detection operations happen in parallel to speed up booting.  A 
side effect is that which nic is initially called eth0/eth1 (etc.) is 
essentially random even if you have modules specified in modprobe.conf 
for certain types.  However, when the configuration is applied as the 
network is started, if the /etc/sysconfig/network-scripts/ifcfg-eth? has 
  HWADDR= values that match, the corresponding interface is renamed and 
used.  If you use the GUI tools to set up the interfaces or they haven'b 
been changed since the initial install, the HWADDR addresses should be 
correct.  If you copy a working machine to a new server or swap NIC 
cards you have to fix the HWADDR to match.  Earlier kernels didn't 
require this because you could count on the device detection to always 
happen in the same order and the matching ifcfg-eth? would be applied 
even if you removed the HWADDR entry.

-- 
   Les Mikesell
    lesmikesell at gmail.com