EL 4.0 and how it does Networking now.

[Rick Stevens]

Waldher, Travis R wrote:
> 
>>-----Original Message-----
>>From: Rick Stevens [mailto:rstevens at vitalstream.com]
>>
>>>2) modules.conf is now modprobe.conf.
>>>	The "below" nomenclature no longer works:
>>>		"below bonding e1000 bcm5700"
>>>	The "options" feature appears to work the same, but I won't know
>>>for 	ure until I get past #3.
>>
>>Yes, for 2.6 kernels, the file is now modprobe.conf.
> 
> 
> Except now that below is no longer a feature that can be used.  I need
> to know that the problems I had 2 years ago(?) with interfaces
> arbitrarily being loaded with the wrong drivers is fixed.
> 
> (Back then you gave me the "below" information.)

"below" simply tells modprobe that bonding uses e1000 and bcm5700.
I don't know that the e1000 entry was necessary, but kudzu likes to
bind the tg3 driver to Broadcom chips when the system is configured.
Essentially anything you could do with "above" or "below" can be
accomplished using modprobe.conf's "install" command.  See my comments
further on (I almost said "below" :-)) for details.

>>>3) The bonding driver.  Apparently, someone decided that no one
> 
> would
> 
>>>ever want to use more than 1 interface.  So they changed it to only
>>>support one.  When you attempt to bring up the second bonding
> 
> interface,
> 
>>>you get this:
>>>
>>>	"bonding1 device bond1 does not seem to be present, delaying
>>>initialization."
>>>
>>>There is an entry in
>>>/usr/src/kernels/2.6.9-5.EL-smp-i686/include/linux/if_bonding.h that
>>>controls this.
>>
>>Yeah, I see that.  Odd.
> 
> 
> 
> Better yet, I looked at an EL 3.0 system, the if_bonding.h file includes
> the same entries.  It appears I may have focused on the wrong problem
> (the header file) or something else has been modified and that setting
> now has relevance.

Hmmmm.  I dunno.  Actually, some time this week I'll be configuring up
some systems that will require two bonding interfaces.  These will be
done on both ES3 (2.4 kernel) and CentOS 4 (2.6 kernel--essentially a
whitebox ES4 distribution) systems and I'll get some hands-on with
those.

> So far Redhat's answer is... either a mis-configuration on my end, which
> is possible, but IMO doubtfull.  Unless features changed and weren't
> documented otherwise it's a bug in the module.  "recompiling it should
> not be an option".

Well, humph!  You should have stuff such as this:

/etc/sysconfig/network-scripts/ifcfg-bond0:
	DEVICE=bond0
	BOOTPROTO=static
	IPADDR=www.xxx.yyy.zzz
	NETMASK=255.255.255.0
	ONBOOT=yes

/etc/sysconfig/network-scripts/ifcfg-bond1:
	DEVICE=bond01
	BOOTPROTO=static
	IPADDR=aaa.bbb.ccc.ddd
	NETMASK=255.255.255.0
	ONBOOT=yes

/etc/sysconfig/network-scripts/ifcfg-eth0:
	DEVICE=eth0
	MASTER=bond0
	SLAVE=yes
	BOOTPROTO=none
	ONBOOT=yes
	TYPE=Ethernet

/etc/sysconfig/network-scripts/ifcfg-eth1:
	DEVICE=eth1
	MASTER=bond0
	SLAVE=yes
	BOOTPROTO=none
	ONBOOT=yes
	TYPE=Ethernet

/etc/sysconfig/network-scripts/ifcfg-eth2:
	DEVICE=eth2
	MASTER=bond1
	SLAVE=yes
	BOOTPROTO=none
	ONBOOT=yes
	TYPE=Ethernet

/etc/sysconfig/network-scripts/ifcfg-eth3:
	DEVICE=eth3
	MASTER=bond1
	SLAVE=yes
	BOOTPROTO=none
	ONBOOT=yes
	TYPE=Ethernet

Then, in /etc/modprobe.conf:

     alias eth0	bcm5700
     alias eth1	bcm5700
     alias eth2	e1000
     alias eth3	e1000
     install bond0 /sbin/modprobe -a eth0 eth1 && /sbin/modprobe bonding
     alias bond0	bonding
     options bond0	mode=x,miimon=100,primary=eth0,etc.
     install bond1 /sbin/modprobe -a eth2 eth3 && /sbin/modprobe bonding
     alias bond1	bonding
     options bond1	mode=x,miimon=100,primary=eth2,etc.

You get the idea.  The "install" lines above the "alias bondX" lines
force the ethernet modules to load before the bonding driver does, and
this is how you accomplish what "below" used to do.  If you look at the
logic, it makes up a shell pipeline where the "modprobe eth0 eth1" has
to be done BEFORE the "modprobe bonding" is run.  Ah, the logical "&&"
operator rears its ugly-but-useful head!  Note also that the bonding
driver won't load unless drivers for BOTH cards that make up the bond
are loaded.  If either fails, bondX won't be created.

BTW, the reason you may have to use the install command is that the
"ifup" commands are run based on the ASCII sort order of the
"ifcfg-xxxx" filenames in /etc/sysconfig/network-scripts, and "bondX"
comes before "ethX", right?  (NOTE TO Red Hat and/or kernel guys:
Someone really oughta fix that ruddy /etc/rc.d/init.d/network script so
bonding devices are brought up last--or at least after all of their
respective slaves are alive!)

FWIW, the current "readme" for bonding should be in

     /usr/src/linux-whatever/Documentation/networking/bonding.txt

and should have all the gory details--yet another good reason to have
the kernel source installed if you have the space to do it.
----------------------------------------------------------------------
- Rick Stevens, Senior Systems Engineer     rstevens at vitalstream.com -
- VitalStream, Inc.                       http://www.vitalstream.com -
-                                                                    -
-  You know you've landed gear-up when it takes full power to taxi.  -
-                                                -- Chuck Yeager     -
----------------------------------------------------------------------