[Beowulf] fast interconnects

[Peter Kjellstrom]

On Tuesday 23 May 2006 15:27, Joachim Worringen wrote:
> Eugen Leitl wrote:
> > Switches (crossbars) don't scale for very many ports. Especially if you
> > have to do cut-through switching at those high speeds. It would be good
> > if each NIC would came with an integrated switch, with enough ports to
> > wire at least a 3d torus (where you route/switch messages via Bresenham).
>
> You know that such NICS do exist for some years? If not, take a look at the
> SCI interfaces at http://www.dolphinics.com. Only a few ns per NIC routing
> latency. However, all HPC switch vendors will tell you that their switches
> scale great, and have full bisection bandwidth. And I guess they are not
> totally wrong.
>
> > In regards to keeping the wires short, does this IBM trick of keeping all
> > wires equal-length work well on 3d lattices, and above? This would seem
> > to be a must for those coming (hopefully) Hypertransport motherboards
> > with connectors.
>
> I assume you refer to the "interleaving" of node connections? Is this an
> IBM patent? It *might* be after all the patents I've seen so far...
>
> The cabling is already quite tricky for 3D setups. I don't think you would
> like to go beyond. There are good reasons why nobody has done it yet.

We have a 5x8x5 SCI 3d torus, it is _NOT_ fun to cable or debug. I much prefer 
my IB switches.

/Peter

>   Joachim

-- 
------------------------------------------------------------
  Peter Kjellstrom               | E-mail: cap at nsc.liu.se
  National Supercomputer Centre  |
  Sweden                         | http://www.nsc.liu.se
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: not available
Url : http://www.scyld.com/pipermail/beowulf/attachments/20060613/41ad247c/attachment.bin