[K12OSN] Re: Networking a new school for K12LTSP?

["Terrell Prudé Jr."]

Robert Arkiletian wrote:
> On 1/31/07, Robert Arkiletian <robark at gmail.com> wrote:
>> On 1/31/07, Petre Scheie <petre at maltzen.net> wrote:
>> > Terrell Prudé Jr. wrote:
>> > > Robert Arkiletian wrote:
>> > >> On 1/29/07, Joseph Bishay <joseph.bishay at gmail.com> wrote:
>> > >>> Hello,
>> > >>>
>> > >>> I hope you are doing well.
>> > >>>
>> > >>> Thank you all for the comprehensive reply!
>> > >>>
>> > >>> Once I started reading your email, I realized that probably the
>> best
>> > >>> way to proceed was to work with the idea of NIC Bonding or port
>> > >>> trunking.  I have a surplus of Gigabit cards so I could put 3 in a
>> > >>> server (reading online I found that more than 3 wasn't going to
>> give
>> > >>> enough of an improvement due to the PCI bus limitations -- can
>> anyone
>> > >>> validate this?) and then send all 3 of those to the switch. I
>> could
>> > >>> then bond 3 ports from that switch to the next one (we'll probably
>> > >>> have 2 x48 gigabit switches for the whole building -- still
>> counting
>> > >>> the number of ports/computers required) so as to deal with the
>> > >>> bandwidth.  The cost of some of those fiber <-> copper converts
>> look
>> > >>> rather daunting.
>> > >>>
>> > >>> I would VERY MUCH prefer to use only 1 server for the entire
>> building
>> > >>> -- I am still very much a novice at this and the complexities of
>> > >>> setting up multiple servers or splitting into application &
>> /home with
>> > >>> LAPD sounds rather daunting.
>> > >>>
>> > >> If your still set on one server also have a look at this
>> > >> http://k12ltsp.org/mediawiki/index.php/Technical:Subnetting
>> > >> Instead of port trunking I think this would be a better idea.
>> > >> Especially if you are going to have 2 48 port switches that
>> could be
>> > >> on different gigabit linked subnets.
>> > >
>> > > Hmm...I hadn't thought of that particular application
>> myself--addressing
>> > > bandwidth bottlenecks--but you're right, that sure would do it! 
>> That
>> > > never even occurred to me...thanks!
>> > >
>> > > --TP
>> > I recall reading somewhere that three gigabit cards is probably the
>> max that the PCI bus
>> > can handle.  Can anyone confirm or deny this?
>>
>> No. A gigabit card is 1 Gibabit/s (that's 1 billion bits per second).
>> Each byte is 8 bits. So  it maxs out at 125MB/s. A simple PCI bus can
>> handle 133MB/s max. So 1 gigabit ethernet card can saturate a PCI bus
>
> Correction:
> PCI 2.2 spec is 32 bits at 66Mhz which equals 266MB/s.  So 2 gigabit
> nics should be able to saturate it. The original PCI bus was 32bits at
> 33Mhz which is 133MB/s.
>

True, but if your PCI bus is 64-bits at 66MHz (i. e. PCI-X), then you're
fine, as you then have 532MB/s.  I've always been sure to buy 64-bit,
66MHz NIC's for this reason.  Same with RAID cards; PCI-X whenever possible.

--TP