Gigabit ethernet or myrinet ?

[Greg Lindahl]

On Thu, Feb 22, 2001 at 11:13:18AM +0800, Woo Chat Ming wrote:

>   I am going to set up a 100-nodes beowulf cluster
> to do scientific simulation. Is Gigabit ethernet or
> Myrinet better ? Does anyone has performance comparation
> of them ?

The main reason to choose one or another isn't necessarily
performance. Here's a summary:

Myrinet:
cards expensive, big switches possible and relatively cheap
mpich/gm gives near peak performance

Gigabit Ethernet over TCP/IP:
cards cheap, big switches expensive and annoying
mpich over tcp/ip gives 1/2 peak performance and eats a lot of cpu

Gigabit Ethernet with MVIA driver:
cost is the same, but only a few cards work with MVIA
mvich (mpich + vi device) is fairly new, and isn't really ready
  for prime time yet. (If I'm wrong about this, someone please
  correct me.)

Try pricing a 100-port gigabit switch with good bisection bandwidth.
This just happens to be near the sweet spot for Myrinet.

-- greg