[Beowulf] torus versus (fat) tree topologies

[Joachim Worringen]

Chris Sideroff wrote:
> My investigation thus far has led me to believe that one reason a torus 
> topology might be better is because it eliminates the need for a switch. 
>  On the other hand fat tree interconnects seem to dominate the 
> larger(est) clusters out there, why?
> 
> I'm looking more for comments about implementation and maintenance 

Torus topologies are easier to scale as cost and effort scale linearly 
with the number of nodes. And there wer and are some very 
popular/well-performing machines using this topology (Intel Paragon/ASCI 
  Red, Cray T3D/E, and now BlueGene). In the cluster range, you can use 
SCI ('Wulfkit') for this.

FatTree topology gives better (theoretically full) bisection bandwidth, 
and examples like Quadrics and Myrinet show that this can be done in a 
(more or less) cost-effective way.

Manageability of a fattree network should be easier, as a failing node 
does not affect message routing at all. Which doesn't mean that the 
difference in complexity concerning this is necessarily experienced by 
the user or administrator if the management software is well implement.

Not to forget that other topologies do exist, too (EarthSimulator/NEC SX 
use a full single-stage crossbar).

  Joachim

-- 
Joachim Worringen - NEC C&C research lab St.Augustin
fon +49-2241-9252.20 - fax .99 - http://www.ccrl-nece.de