very high bandwidth, low latency manner?

[Steffen Persvold]

On Thu, 4 Apr 2002, Jim Lux wrote:

> What's high bandwidth?
> What's low latency?
> How much money do you want to spend?
>
> Ethernet is cheap, $100-$200/node for 100 Mbps or GBE (by the time you get
> switches, cables, adapters, etc.)
> Latency is kind of slow (compared to dedicated point to point links)
>
>

Well this is a "touchy" topic since different people has different
opinions. There is also different ways of measuring bandwidth mainly
point to point (two machines talking together) and bisection (dividing
your nework in half and let the one half talk to the other which kind of
shows how the network scales with more nodes).

Also some people like to talk about the hardware bandwidth and hardware
latency, while the thing that really matters (IMHO) is application to
application bandwidth and latency.

I don't want to start a flamewar here, but I _think_ (not knowing real
numbers for other high speed interconnects) that SCI has atleast the
lowest latency and maybe also the highest point to point bandwidth :

SCI application to application latency   : 2.5 us
SCI application to application bandwidth : 325 MByte/sec

Note that these numbers are very chipset specific (as most high speed
interconnect numbers are), these numbers are from IA64. Here are numbers
from a popular IA32 platform, the AMD 760MPX :

SCI application to application latency   : 1.8 us
SCI application to application bandwidth : 283 MByte/sec


More "real" performance numbers using MPI over SCI (also collective and
application benchmarks) can be located on Dolphin's homepage
http://www.dolphinics.com

Other popular high speed interconnects I know of is Myrinet (considered
the main competitor to SCI for cluster interconnects) and Giganet. There
are some performance numbers on Myricoms homepage  (http://www.myricom.com)
but I doubt if that is for their latest hardware generation (correct me if
I'm wrong).

Best regards,
-- 
  Steffen Persvold   | Scalable Linux Systems |   Try out the world's best
 mailto:sp at scali.com |  http://www.scali.com  | performing MPI implementation:
Tel: (+47) 2262 8950 |   Olaf Helsets vei 6   |      - ScaMPI 1.13.8 -
Fax: (+47) 2262 8951 |   N0621 Oslo, NORWAY   | >320MBytes/s and <4uS latency