[K12OSN] nic bonding anyone using it in k12ltsp labs

[James P. Kinney III]

On the APS rollout, we used bonding as follows:

Each server came with dual Gbit nics onboard. A pair of dual Gbit nics
was added . To retain the actual throughput required, the add-on cards
were installed in the PCI-X slots at 133MHz. The 100MHz slot can't
sustain two full Gbit connections simultaneously.

The onboard nics were used for connection to the NAS server and
connection to the internet facing environment. The 4 add-on nics were
bonded using mode-5 (transmission load balancing) to form the ethernet
device bond0 (comprised of eth2-5).

Note: The start-up ordering is essential to avoid problems.
In /etc/modprobe.conf each nic gets it's own listing (alias eth2
eepro1000, etc) but be sure the bond0 listing is _AFTER_ all of the
other nics are initialized. If not, the bond may be comprised of the
incorrect nics.

Since each server had a typical client count of 100, the bonding was
essential. With 4 bonded Gbit lines we had an effective 3Gbit continuous
data stream (ethernet eat about 25% of the total in packet overhead with
100Mb connections) feeding 80-120 clients per server.

Network bandwidth was never been the issue with this setup. It has still
had out of memory issues (8GB only per server is barest minimum. I
learned a valuable lesson - when the bean counter asked what was the
minimum requirements I shouldn't have told them the _engineering_
minimum as based on testing. I should have told them the practical
minimum based on anticipated maximum loading of all machines running
full load of flash enabled clients with java apps!) as well as other
hardware issues.

On Wed, 2007-11-28 at 21:23 -0600, Barry Cisna wrote:
> Hello List,
> 
> Just wondering if anyone at current is running any k12ltsp labs with nic
> bonded nics? If so what if any is the actual performance boost? There are
> a few how to's out now that makes nic bonding seem pretty reliable. It
> seems you would bound to benefit some with " more pipes":) I'm not enough
> of a numbers cruncher/ bean counter to figure were the actual bottleneck
> lies as far as bus/pci speed etc. Would there be any disadvantages of
> using a quad nic versus two or four nics for example? I thought if a
> person plopped in a quad nic into a server you could use two holes for
> eth0 and two holes for eth1. I've always wanted to toy around with this
> but it seems your better off finding what happens from actual past users
> of this kinda stuff.
> 
> Thanks,
> 
> Barry Cisna
> westcentral school
> 
> _______________________________________________
> K12OSN mailing list
> K12OSN at redhat.com
> https://www.redhat.com/mailman/listinfo/k12osn
> For more info see <http://www.k12os.org>
> 
-- 
James P. Kinney III          
CEO & Director of Engineering 
Local Net Solutions,LLC        
770-493-8244                    
http://www.localnetsolutions.com

GPG ID: 829C6CA7 James P. Kinney III (M.S. Physics)
<jkinney at localnetsolutions.com>
Fingerprint = 3C9E 6366 54FC A3FE BA4D 0659 6190 ADC3 829C 6CA7
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: <http://listman.redhat.com/archives/k12osn/attachments/20071129/e2139169/attachment.sig>