[Beowulf] Clos networking and myrinet

[Mack.Joseph at epamail.epa.gov]

Joseph Mack PhD, High Performance Computing & Scientific Visualisation
LMIT, Supporting the EPA Research Triangle Park, NC 919-541-0007 Federal
Infrastructure Contact-Ravi Nair 919-541-5467 - nair.ravi at epa.gov,
Federal Visualization  Contact - Joe Retzer, Ph.D. 919-541-4190 -
retzer.joseph at epa.gov

beowulf-bounces at beowulf.org wrote on 06/08/2005 02:02:40 PM:

> On Tue, Jun 07, 2005 at 11:33:06AM -0400, Mack.
> Joseph at epamail.epa.gov wrote:
>
> > >From other reading I find that 3 layer Clos networks
> > are blocking but that 5 (and above) layers are non-blocking.
>
> That blocking and non-blocking stuff is all a myth. Clos
> networks have a special property if you are doing circuit
> switching, like is done in the telecom market: Node A
> talks to Node B and only node B. With a Clos network,
> you have a dedicated path from A to B for
> every node on the network.
>
> In a Linux cluster, each node talks to multiple other nodes. So they
> always interfere with each other.

will have to think about this. Is this true for 5, 7 ... n
layer Clos networks too?

Since I don't understand what's going on here, I'll try a few
questions

o If all nodes in a beowulf were exchanging data in pairs
before a barrier and then later were exchanging data in
a different combination of pairs before another barrier,
which pairwise exchanges would be blocked?

o Is a Clos network ever non-blocked?

o Why does myrinet use Clos switching for beowulfs, if
block/non-blocking isn't an issue?


> > With large beowulfs, why is the diameter of the network
> > increased from 3 to 5 to 7...?
>
> Because the maximum number of nodes you can support with
> only 3 layers is limited by the size of the switch chips
> you're using.

Got it (I was expecting something much more complicated).

Thanks Joe