[Beowulf] SC|05: will 10Ge beat IB?

[Olli-Pekka Lehto]

ctierney at hypermall.net wrote:
> Quoting Mark Hahn <hahn at physics.mcmaster.ca>:
> 
> 
>>last year, there was lots of buzz around IB as an up-and-comer,
>>with some undercurrents of doubt related to latency, memory 
>>consumption on large nets, proliferation of stacks, etc.
>>
>>this year, the picture was a lot clearer, with at least some 
>>progress on all these.  lots of vendors pushing IB, including 
>>some starting to push IB for storage.  IB-storage vendors seem
>>to be regarding it as a faster FC, though - to me, that's a huge
>>step backwards, since FC was never really inter-operable, at least
>>not in the sense that ethernet is inter-op.
>>
>>further, there was also LOTS of 10G ethernet this year, and I have 
>>to wonder: will 10G squeeze IB out of the market?  
> 
> 
> That depends.  Do you think that iSCSI (or some other variant) is finally
> going to take off as it was supposed to?  IB has some QoS features that
> may be important when using IB for multiple purposes, like HSN and FC.
> I don't know if that is true, just a theory.
> 
> Also, 10G is still quite expensive (except Myrinet 10g).  
> Fibre-media converters are still expensive and as far as I have heard, no
> one is pushing 10G over copper.  IB is quite cheap compared to 10G, but
> it is a new technology and adoption outside of HPC is slow.  The volume
> isn't there yet to bring pricing where it should be.
> 

Chelsio's T210-CX4 TOE NIC has been around for about a year. Like the 
Myricom product it utilizes the 10GBase-CX standard. It essentially 
pushes 10GbE over InfiniBand cabling directly using the XAUI interface 
(normally used for on-PCB communication on 10G devices).

There are also several vendors which provide interface modules for 
switches. Of course with 10Gbase-CX you lose the advantage of common and 
relatively cheap UTP cabling and are limited to 50ft per link. It seems 
to me to be a stopgap standard that will be overtaken by 10Gbase-T in a 
few years time.

I don't have a quote on the pricing but it should be competitive with 
IB. If anyone insight on the economics and performance of 10GBase-CX vs. 
IB in cluster applications I'd be interested to hear it.

Olli-Pekka
-- 
Olli-Pekka Lehto, Systems Specialist, Systems Services, CSC
PO Box 405 02101 Espoo, Finland; tel +358 9 457 2215, fax +358 9 4572302
CSC is the Finnish IT Center for Science, www.csc.fi,
e-mail: Olli-Pekka.Lehto at csc.fi