[Beowulf] using two separate networks for different data streams

[Greg Lindahl]

On Mon, Jan 30, 2006 at 07:32:37AM -0700, Keith D. Underwood wrote:

> The physical layer on a link by link basis
> doesn't even have the same signaling rate as IB (3.2 Gb/s/wire pair for
> XT3 vs. 2.5 Gb/s/wire pair for IB).

We found that most serdes you could buy that do 2.5 also do 3.125,
because they're actualy PCI express serdes.

> XT3 is 12 bits wide and I'm not sure if anybody has done a 12X IB
> yet.

The Mellanox switch silicon does 12X, and it is sometimes used that
way to reduce cable counts. Sun sells a 8-port 12X switch.

I got several emails from people saying "Don't you mean the Cray XD1?"
The XD1 seems to be similar to InfiniPath in that it emits packets
that can be switched by a normal InfiniBand switch. But I haven't
heard anything about Cray doing the software to run normal IB
protocols.

-- greg