[Fwd: Re: 32-port gigabit switch]

[Mark Hahn]

> > >clusters.  Remember on clusters latency is more important than
> > >throughput. 
> > 
> > Bzzt. Wrong. Dead Wrong. "It depends" is the correct answer. (Don't
> > pass GO, don't collect $200).

it does depend.  
but as generalizations go, bandwidth is easy; latency is hard.

> The grid stuff, like Mark Hahn described, only works on small clusters
> and increases the hardware requirements by a huge factor (2-3 cards per
> system and multiple switches). 

it's very beowulfish to take advantage of changing hardware.  
many/most server boards these days have two builtin gigabit links.  
and why not take advantage of the new crop of small/cheap gige switches?
pricewatch has 8-ports for $US 320!

> Plus the additional admin time to
> configure and maintain all the networking options.  

wiring requires some patience and care; I don't really see why admin
would be hard.  in fact, I think for the grid-of-switches approach, the 
builtin spanning tree would work fine, since there are no redundant links,
unlike with a fat tree.  you'd still have to set routes on each node,
but wouldn't that just take a pretty simple perl script?

> Its a nightmare.  Go with cheap, used Myrinet and save yourself a lot of
> heartache.

I don't see myrinet (or quadrics or IB) at <$100/port any time soon.