[Beowulf] fast interconnects

[Jim Lux]

At 09:21 PM 5/22/2006, Mark Hahn wrote:


> > 10GigE people need to solve problems that do not exist in IB, and the
> > 10GigE solutions for those issues will results in more expensive NICs.
>
>but it's silly to only focus on nic prices.  I fully expect a myrinet
>10gbase-t nic to cost at least around $500.  that sounds like a lot more
>than a memfree IB nic, but if IB switch ports cost more than 10gbase-t
>ports, it may be irrelevant.  besides, I'd actually prefer to pay a bit
>more for my nics - to put intelligence in the nic rather than the switch.

There's sort of three problems that need to be solved for really fast links:

1) The actual transport over the wire. To get the high speeds, you'll need 
intelligence on both ends of the wire (adaptive EQ and coding), so I would 
expect that would drive costs on both the NIC and switch end, roughly 
equally (unless there's some sort of subsidization going on)

2) Actually getting data out of the PC into the wire at 10 Gbps is no 
trivial matter.  66 Megatransfers per second at 32 bits/transfer is only 2 
Gbps.  I suspect that systems that can make effective use of these sorts of 
rates will NOT be based on "cards that plug into a backplane".

3) Then, there's the switch.  If it's a *switch* that sets up circuits, 
then it can be fairly inexpensive.  If it's something that has to do packet 
buffering, it gets a lot more expensive.

Jim