[K12OSN] bonding help...sos!!!

Wed Feb 25 14:29:02 UTC 2004

well, we experienced some kind of corruption with a stable Dell 
Poweredge 6650, and in desperation to fix it,
we upgraded to Fedora Core1.
we thought we had the problems all licked, but at 5:45 this morning, we 
had a kernel panic.
It is now 1:09, and we have finally discovered through trial, and error, 
that it is indeed our bonding that is causing all of the problems.

We have 4 bcm5700's in, and we tried different kernels, all 3 current 
fedora kernels,
a redhat kernel(which wouldn't even find the root filesystem even though 
it did last week)
and even a fedora core2 kernel.  We tried bcm5700 drivers from broadcom 
website, along with bcm5700 from 3com,
all gave the same fatal exceptions, or cpu exception, or just a plain 
old hard freeze.
We even saw some tcp.c errors, and kernel assertion errors.

Finally we compiled our own, compiled tg3 support into the kernel versus 
module support.
That includes bonding also.

No matter if we used a 2.4 kernel, or 2.6, or compiled our own,
or used the bcm5700 linux driver...
bonding always hosed the system.

We can only keep the server up in single eth0 mode, no bonding.

we have even read the bonding.txt file in Documentation/networking in 
the kernel source,
and we compiled ifenslave.c ourselves, linked against our new 2.6.3 
kernel sourcetree using gcc296

when we try to start bonding, or ifup bond0,
we get all kinds of errors.
it doesn't matter if we are using the kernel with tg3 compiled in, or not,
we get the same errors.
such as ifenslave eth0 failed,
cannot get speed on eth0,  forcing 100

has bonding changed in fedora?
the bonding.txt did not reflect any changes in our config 
scripts(/etc/sysconfig/network-scripts)
we have a ifcfg-bond0, with all of the configurations set,
then ifcfg-eth0, ifcfg-eth1, ifcfg-eth2 all set as slaves to bond0

I can give more detailed errors after the end of the day, and everyone 
is off the server again.
We are limping along on one nic.

Any suggestions are very welcome....even if they include holy water :-)

here is our bonding config

[root at samba bond]# for i in $(ls); do echo "config file $i:"; cat $i; done
config file ifcfg-bond0:
DEVICE=bond0
USERCTL=no
ONBOOT=yes
BOOTPROTO=none
BROADCAST=10.10.255.255
NETWORK=10.10.0.0
NETMASK=255.255.0.0
IPADDR=10.10.10.2
GATEWAY=10.10.10.1config file ifcfg-eth0:
USERCTL=no
BOOTPROTO=none
DEVICE=eth0
ONBOOT=yes
MASTER=bond0
SLAVE=yes
config file ifcfg-eth1:
USERCTL=no
BOOTPROTO=none
DEVICE=eth1
ONBOOT=yes
MASTER=bond0
SLAVE=yes
config file ifcfg-eth2:
USERCTL=no
BOOTPROTO=none
DEVICE=eth2
ONBOOT=yes
MASTER=bond0
SLAVE=yes
config file ifcfg-eth3:
USERCTL=no
BOOTPROTO=none
DEVICE=eth3
ONBOOT=yes
MASTER=bond0
SLAVE=yes

here is our modules.conf for 2.4 kernels
[root at samba etc]# cat modules.conf
alias eth0 tg3
alias eth1 tg3
alias eth2 tg3
alias eth3 tg3
alias scsi_hostadapter aic7xxx
alias scsi_hostadapter1 megaraid
alias bond0 bonding
#alias bond0 miimon=100

here is modprobe.conf for 2.6 kernels

[root at samba etc]# cat modprobe.conf
include /etc/modprobe.conf.dist
alias eth0 tg3
alias eth1 tg3
alias eth2 tg3
alias eth3 tg3
alias scsi_hostadapter aic7xxx
alias scsi_hostadapter1 megaraid
alias bond0 bonding
[root at samba etc]# cat modprobe.conf.dist
# default modutils aliases
alias binfmt-204 binfmt_aout
alias binfmt-263 binfmt_aout
alias binfmt-264 binfmt_aout
alias binfmt-267 binfmt_aout
alias binfmt-387 binfmt_aout
alias binfmt-332 iBCS
alias binfmt--310 binfmt_java
alias block-major-1 rd
alias block-major-2 floppy
alias block-major-3 ide-probe-mod
alias block-major-7 loop
alias block-major-8 sd_mod
alias block-major-9 md
alias block-major-11 sr_mod
alias block-major-13 xd
alias block-major-15 cdu31a
alias block-major-16 gscd
alias block-major-17 optcd
alias block-major-18 sjcd
alias block-major-20 mcdx
alias block-major-22 ide-probe-mod
alias block-major-23 mcd
alias block-major-24 sonycd535
alias block-major-25 sbpcd
alias block-major-26 sbpcd
alias block-major-27 sbpcd
alias block-major-29 aztcd
alias block-major-32 cm206
alias block-major-33 ide-probe-mod
alias block-major-34 ide-probe-mod
alias block-major-37 ide-tape
alias block-major-44 ftl
alias block-major-46 pcd
alias block-major-47 pf
alias block-major-56 ide-probe-mod
alias block-major-57 ide-probe-mod
alias block-major-58 lvm-mod
alias block-major-88 ide-probe-mod
alias block-major-89 ide-probe-mod
alias block-major-90 ide-probe-mod
alias block-major-91 ide-probe-mod
alias block-major-93 nftl
alias block-major-97 pg
alias block-major-113 viocd
alias char-major-4 serial
alias char-major-5 serial
alias char-major-6 lp
alias char-major-9 st
alias char-major-10-0 busmouse
alias char-major-10-2 msbusmouse
alias char-major-10-3 atixlmouse
alias char-major-10-130 wdt
alias char-major-10-131 wdt
alias char-major-10-135 rtc
alias char-major-10-139 openprom
alias char-major-10-144 nvram
alias char-major-10-157 applicom
alias char-major-10-175 agpgart
alias char-major-10-184 microcode
alias char-major-10-200 tun
alias char-major-10-250 hci_vhci
alias char-major-13 input
alias char-major-13-0 joydev
alias char-major-13-32 mousedev
alias char-major-14 soundcore
alias char-major-19 cyclades
alias char-major-20 cyclades
alias char-major-21 sg
alias char-major-22 pcxx
alias char-major-23 pcxx
alias char-major-27 zftape
alias char-major-34 scc
alias char-major-35 tclmidi
alias char-major-36 netlink
alias char-major-37 ide-tape
alias char-major-48 riscom8
alias char-major-49 riscom8
alias char-major-57 esp
alias char-major-58 esp
alias char-major-63 kdebug
alias char-major-90 mtdchar
alias char-major-96 pt
alias char-major-99 ppdev
alias char-major-107 3dfx
alias char-major-108 ppp_generic
alias char-major-109 lvm-mod
alias char-major-161 ircomm-tty
alias char-major-171 raw1394
alias char-major-195 NVdriver
alias char-major-200 vxspec
alias char-major-206 osst
alias char-major-216 rfcomm
alias dos msdos
alias dummy0 dummy
alias dummy1 dummy
alias iso9660 isofs
alias md-personality-1 linear
alias md-personality-2 raid0
alias md-personality-3 raid1
alias md-personality-4 raid5
alias md-personality-7 multipath
alias net-pf-1 unix
alias net-pf-2 ipv4
alias net-pf-4 ipx
alias net-pf-5 appletalk
alias net-pf-15 af_key
alias net-pf-17 af_packet
alias net-pf-31 bluez
alias netalias-2 ip_alias
alias irlan0 irlan
alias irda-dongle-0 tekram
alias irda-dongle-1 esi
alias irda-dongle-2 actisys
alias irda-dongle-3 actisys
alias irda-dongle-4 girbil
alias irda-dongle-5 litelink
alias irda-dongle-6 airport
alias irda-dongle-7 old_belkin
alias bt-proto-0 l2cap
alias bt-proto-2 sco
alias bt-proto-3 rfcomm
alias bt-proto-4 bnep
alias plip0 plip
alias plip1 plip
alias tunl0 ipip
alias cipcb0 cipcb
alias cipcb1 cipcb
alias cipcb2 cipcb
alias cipcb3 cipcb
alias ppp0 ppp
alias ppp1 ppp
alias slip0 slip
alias slip1 slip
alias tty-ldisc-1 slip
alias tty-ldisc-3 ppp_async
alias tty-ldisc-11 irtty
alias tty-ldisc-14 ppp_synctty
alias tty-ldisc-15 hci_uart
alias ppp-compress-18 ppp_mppe
alias ppp-compress-21 bsd_comp
alias ppp-compress-24 ppp_deflate
alias ppp-compress-26 ppp_deflate
alias parport_lowlevel parport_pc
alias usbdevfs usbcore
alias xfrm-type-2-50 esp4
alias xfrm-type-2-51 ah4
alias xfrm-type-2-108 ipcomp
alias xfrm-type-10-50 esp6
alias xfrm-type-10-51 ah6
alias xfrm-type-10-108 ipcomp6
alias cipher_null crypto_null
alias digest_null crypto_null
alias compress_null crypto_null
alias sha384 sha512
options sb io=0x220 irq=7 dma=1 dma16=5 mpu_io=0x330
install binfmt-0000 /bin/true
install binfmt_misc /sbin/modprobe --ignore-install binfmt_misc && { 
/bin/mount -t binfmt_misc none /proc/sys/fs/binfmt_misc > /dev/null 2>&1 
|| :; }
install char-major-10 /bin/true
install char-major-10-1 /bin/true
install dummy0 /sbin/modprobe -o dummy0 --ignore-install dummy
install dummy1 /sbin/modprobe -o dummy1 --ignore-install dummy
install eth0 /bin/true
install hid /sbin/modprobe --ignore-install hid && { /sbin/modprobe 
keybdev; /sbin/modprobe mousedev; /bin/true; }
install net-pf-10 /bin/true
install net-pf-19 /bin/true
install net-pf-3 /bin/true
install net-pf-6 /bin/true
install ov518_decomp { /sbin/modprobe ov511; } ; /sbin/modprobe 
--ignore-install ov518_decomp
install scsi_hostadapter /bin/true
install usbmouse /sbin/modprobe --ignore-install usbmouse && { 
/sbin/modprobe hid; /bin/true; }
install wacom /sbin/modprobe --ignore-install wacom && { /sbin/modprobe 
evdev; /bin/true; }
remove binfmt_misc { /bin/umount /proc/sys/fs/binfmt_misc > /dev/null 
2>&1 || :; } ; /sbin/modprobe -r --ignore-remove binfmt_misc
remove hid { /sbin/modprobe -r keybdev; /sbin/modprobe -r mousedev; } ; 
/sbin/modprobe -r --ignore-remove hid
remove ov518_decomp /sbin/modprobe -r --ignore-remove ov518_decomp && { 
/sbin/modprobe -r ov511; /bin/true; }
remove usbmouse { /sbin/modprobe -r hid; } ; /sbin/modprobe -r 
--ignore-remove usbmouse
remove wacom { /sbin/modprobe -r evdev; } ; /sbin/modprobe -r 
--ignore-remove wacom