[Beowulf] Cheap SDR IB

[Gilad Shainer]

> > Just in case you've missed the announcements:
> > 
> > http://www.clustermonkey.net//content/view/222/1/
> 
> Interesting that Infiniband is getting down close to the 
> predicted $100 per HCA price.  Certainly $250 per node (HCA, 
> cable, and switch) for 24 ports makes it easier to justify if 
> the performance is there.  So of course that begs the 
> question, what is the performance?
> 
> I noticed that the price is for the InfiniHost III Lx, does 
> anyone know if the Ex (mentioned in the performance table) 
> and Lx (mentioned in the price list) perform identically?  Or 
> any of the standard numbers for MPI latency and/or bandwidth 
> for the Lx part?
> 
> I googled around and found a few PDFs, but no real numbers.  
> Some EE magazine mentioned 4us roughly, but it wasn't clear 
> exactly what they were measuring.
> 

Mellanox has 4 IB families - InfiniHost, InfiniHost III Lx, InfiniHost
III Ex and ConnectX. InfiniHost is a PCI-X device and the other 3 are
PCIe based. The Ex and LX support IB DDR and ConnectX supports IB QDR
and PCIe Gen2.
>From latency point of view, the LX is a 3.5-3.3us MPI latency (SDR/DDR),
the Ex is 2.6-2.3us  (SDR/DDR) and ConnectX is 1.2-1us (PCIe Gen1/Gen2).
For BW, Lx provides ~1400MB/s, EX is ~1500MB/s and ConnectX is ~1900MB/s
uni-directional on PCIe Gen2.  

Feel free to contact me directly for more info.

Gilad.