[K12OSN] nic bonding anyone using it in k12ltsp labs

[Jim Kronebusch]

I'll just back up James' info below.  I did use bonding a few times for k12ltsp labs and
it worked great.  I wouldn't bother bonding for the internet side, not enough traffic. 
And in the research for my new server I somehow concluded that a quad card would
overload any PCI bus.  So the max I would recommend as well is a dual as long as it is a
PCI-X (I think PCI-E was capable as well).  Otherwise just stick with single nic cards
if you are unsure.  I have had the best luck with Intel Pro1000 cards.  Everything
recognizes them no matter what distro I use and they fully support all the bonding
features.  Be careful before ordering the Server nic series by making sure your machine
can handle full length cards.  Otherwise the way the bonding works is you can mix and
match speeds and models of cards, so you should be able to play around with whatever you
have lying around.

I am going to try the tux math test today when a couple of our labs clear out.

Jim

On Thu, 29 Nov 2007 07:29:49 -0500, James P. Kinney III wrote
> On the APS rollout, we used bonding as follows:
> 
> Each server came with dual Gbit nics onboard. A pair of dual Gbit nics
> was added . To retain the actual throughput required, the add-on cards
> were installed in the PCI-X slots at 133MHz. The 100MHz slot can't
> sustain two full Gbit connections simultaneously.
> 
> The onboard nics were used for connection to the NAS server and
> connection to the internet facing environment. The 4 add-on nics were
> bonded using mode-5 (transmission load balancing) to form the ethernet
> device bond0 (comprised of eth2-5).
> 
> Note: The start-up ordering is essential to avoid problems.
> In /etc/modprobe.conf each nic gets it's own listing (alias eth2
> eepro1000, etc) but be sure the bond0 listing is _AFTER_ all of the
> other nics are initialized. If not, the bond may be comprised of the
> incorrect nics.
> 
> Since each server had a typical client count of 100, the bonding was
> essential. With 4 bonded Gbit lines we had an effective 3Gbit continuous
> data stream (ethernet eat about 25% of the total in packet overhead with
> 100Mb connections) feeding 80-120 clients per server.
> 
> Network bandwidth was never been the issue with this setup. It has still
> had out of memory issues (8GB only per server is barest minimum. I
> learned a valuable lesson - when the bean counter asked what was the
> minimum requirements I shouldn't have told them the _engineering_
> minimum as based on testing. I should have told them the practical
> minimum based on anticipated maximum loading of all machines running
> full load of flash enabled clients with java apps!) as well as other
> hardware issues.
> 
> On Wed, 2007-11-28 at 21:23 -0600, Barry Cisna wrote:
> > Hello List,
> > 
> > Just wondering if anyone at current is running any k12ltsp labs with nic
> > bonded nics? If so what if any is the actual performance boost? There are
> > a few how to's out now that makes nic bonding seem pretty reliable. It
> > seems you would bound to benefit some with " more pipes":) I'm not enough
> > of a numbers cruncher/ bean counter to figure were the actual bottleneck
> > lies as far as bus/pci speed etc. Would there be any disadvantages of
> > using a quad nic versus two or four nics for example? I thought if a
> > person plopped in a quad nic into a server you could use two holes for
> > eth0 and two holes for eth1. I've always wanted to toy around with this
> > but it seems your better off finding what happens from actual past users
> > of this kinda stuff.
> > 
> > Thanks,
> > 
> > Barry Cisna
> > westcentral school
> > 
> > _______________________________________________
> > K12OSN mailing list
> > K12OSN at redhat.com
> > https://www.redhat.com/mailman/listinfo/k12osn
> > For more info see <http://www.k12os.org>
> > 
> -- 
> James P. Kinney III          
> CEO & Director of Engineering 
> Local Net Solutions,LLC        
> 770-493-8244                    
> http://www.localnetsolutions.com
> 
> GPG ID: 829C6CA7 James P. Kinney III (M.S. Physics)
> <jkinney at localnetsolutions.com>
> Fingerprint = 3C9E 6366 54FC A3FE BA4D 0659 6190 ADC3 829C 6CA7


Jim Kronebusch
Cotter Tech Department
453-5188


-- 
This message has been scanned for viruses and
dangerous content by the Cotter Technology 
Department, and is believed to be clean.