Opteron Vs. Athlon X2
Bill Broadley
bill at cse.ucdavis.edu
Fri Dec 9 01:49:40 UTC 2005
On Thu, Dec 08, 2005 at 05:20:47PM -0800, Bryan J. Smith wrote:
> FYI, I was under the impression that each x1 channel is
> 0.125GBps bi-directional (0.25GBps effective), meaning an x8
> is 1.0GBps bi-directional (2.0GBps effective).
Your impression is wrong. For a pretty reasonable explanation check:
http://en.wikipedia.org/wiki/Pci-e
A few quotes:
First-generation PCIe is often quoted to support a data-rate of 250
MB/s in each direction, per lane. This figure is a calculation from the
physical signalling-rate (2500 Mbaud) divided by the encoding overhead
(10bits/byte.)
...
Note that the datarate already includes the overhead (10 bits per byte).
As for real world practical bandwidth:
Like other high-speed serial interconnect systems, PCIe has significant
protocol and processing overhead. Long continuous unidirectional
transfers (such as those typical in high-performance storage
controllers) can approach >95% of PCIe's raw (channel) data-rate.
Because the channel is full duplex it's easier to get a higher percentage
of the full bandwidth (unlike PCI-x.)
> They kick serious @$$ and prevent all that redundant load
> going back and forth from I/O to memory to CPU back to memory
> and down I/O again, just to do a XOR calculation (the XOR
> itself is nothing, the CPU can do them quickly). If you're
> building a system that is a dedicated storage device, that's
> one thing, and you use software RAID -- but if you're
> building a server that is servicing clients, it's nice to
> keep the storage processing from hogging I/O that could be
> used for network and other services.
We have had this discussion before. Not sure how a few hundred MB/sec
of I/O is supposed to eat up 4GB/sec of Pci-e bandwidth. Keep in mind
that PCI-e isn't shared. To talk to for instance another PCI-e device
you are using seperate lines (again unlike pci-x).
So PCI-e is a point to point duplex connection, reads and writes from
multiple devices do not compete for bandwidth, nor do they have to
negotiate for the bus.
To avoid a repeat of previous arguments please post ACTUAL numbers
showing the superiority of hardware RAID. I don't deny it's possible,
but without real numbers the rest is hand waving. I've sustained well
over a GB/sec of I/O with an opteron system, I've not experienced the
"hogging I/O" problem.
> How are the Linux drivers and user-space support? I've never
The linux drivers seem fine, I've not played with the user-space
tools, so far just the web interface. I know they are out there but
I've not used them. Can anyone else on the list comment?
> used the Areca so I'm very interested. My use of 3Ware
> 7000/8000 series is more about the proven drivers and
> user-space support. I'd be happy to find a PCIe solution
> that is equal to the 3Ware for PCI-64/X.
With software RAID I can't tell the difference between 3ware and areca,
I've don't have any extensive production use of hardware RAID on either.
Not since I lost a few filesystems to a buggy 3ware hardware raid driver
back in the 6800 days. Of course 3ware has gotten much better since then.
> Good to know. But how is the user-space support?
Sorry, no experience.
--
Bill Broadley
Computational Science and Engineering
UC Davis
More information about the amd64-list
mailing list