[Beowulf] Clos networking and myrinet

[Greg Lindahl]

On Tue, Jun 07, 2005 at 11:33:06AM -0400, Mack.Joseph at epamail.epa.gov wrote:

> >From other reading I find that 3 layer Clos networks
> are blocking but that 5 (and above) layers are non-blocking.

That blocking and non-blocking stuff is all a myth. Clos networks have
a special property if you are doing circuit switching, like is done in
the telecom market: Node A talks to Node B and only node B. With a
Clos network, you have a dedicated path from A to B for every node on
the network.

In a Linux cluster, each node talks to multiple other nodes. So they
always interfere with each other.

> With large beowulfs, why is the diameter of the network
> increased from 3 to 5 to 7...?

Because the maximum number of nodes you can support with only 3 layers
is limited by the size of the switch chips you're using. For the
InfiniBand case, this size is 288 ports at 3 layers, because the
building block is a 24-port crossbar switch chip.

-- greg