Microcontrollers/ASICs v. General Purpose Microprocessors -- WAS: Opteron Vs. Athlon X2

[Bryan J. Smith]

On Fri, 2005-12-09 at 12:31 +0100, Eugen Leitl wrote:
> I'm personally not running a cluster, but if you look into the
> Beowulf list archives, people are citing impressive numbers.

Beowulf performance is _inversely_proportional_ to communication load.
Applications scale far more linearly when the slower 10GbE are not used
than the local, HyperTransport interconnects (assuming Opteron).  From
what I've seen, no one is getting anywhere close to 1GBps over layer-2
communication with 10GbE for clustering.  I'm talking about when you're
pushing enough service data around that you need local interconnect,
because 1GBps won't cut it.  Beowulf is not even applicable!

Furthermore, the new, preferred solution is to use HTX (HyperTransport
eXtension) Infinibind cards, which will give you about 1.8GBps real-
world interconnect.  Infiniband's protocol is far more efficient, plus
(in raw transfer terms) it's faster than GbE.  But even then, you don't
use Beowulf for applications where you're taxiing the interconnect
(database, file, etc...).

> SoftRAID easily outperforms these $100-300 controllers. And $1000-1500
> buys you a couple of servers, or a large pile of SATA disks.
> As a semi-hobbyist, it's a pretty simple decision to make.

But not both -- I'm talking about when you are spending about $1,500 on
the combination of server and disk, adding another $300 is a simple
decision to make.  If I just want a 2-disc mirror on a $750-1,000
server, then $100 is also a simple decision to make.  A $1,500-2,000+,
$300 is nothing.

Potent I/O interconnect in a single-socket server -- with server-class
1GbE NIC(s), and a PCI-X or PCIe x4 or x8 slot for storage -- runs you
about $750.  Using a desktop nForce4's PCI by hijacking it's video PCIe
x16 or using its extra PCIe x1 is not server-class -- especially not its
NIC.  Adding a brainless 2-channel 3Ware Escalade 8506-2 adds maybe
$125.  Less than 20%.

If I'm looking at server-class dual-socket, then I'm looking more $1,500
for the system, maybe $2,000+ after disks.  Adding a $300-400 storage
controller is also going to be around 20% increase.  Well worth it in my
experience, especially after MD screw up after MD screw up, especially
when it comes to clients supporting themselves.

-- 
Bryan J. Smith   mailto:b.j.smith at ieee.org
http://thebs413.blogspot.com
------------------------------------------
Some things (or athletes) money can't buy.
For everything else there's "ManningCard."