[Beowulf] Сlos network vs fat tree

[Greg Lindahl]

On Thu, Nov 13, 2008 at 08:54:35PM +0300, Mikhail Kuzminsky wrote:

> I read somewhere that exact evidence of "non-blocking" was performed for 
> Clos networks with >= 3 levels. But most popular Infiniband fat trees has 
> only 2 levels.

Two is three: if you have

   spine  spine
    |   \/  |
   leaf   leaf
   /  \    /  \
node node node node

that's a 3-level network in Clos-speak, because the image you should really
imagine is

node node
  \   /
   leaf
   /  \
spine spine
   \  /
   leaf
   / \
node node

Or, to think about it another way, you have to go through 3 switches
max to get from a node to another. This is also sometimes called the
network diameter.

> (Yes, I know that "non-blocking" for Clos network isn't "absolute" :-))

So useless that it doesn't even help to mention "non-blocking", unless each node
only talks to exactly 1 other node. In which case I won't complain if you mention
the standard "latency" and "bandwidth" benchmarks ;-)

-- greg