Beowulf in a Box

[Mark Brinicombe]

On Mon, 28 Sep 1998, Kragen wrote:

> On Mon, 28 Sep 1998, Bob Drzyzgula wrote:
> > On Sun, Sep 27, 1998 at 08:15:59PM -0400, Kragen wrote:
> > > Well, broadcasting and multicasting are, I think, possible over
> > > Ethernet, but not over PCI.  Also, AFAIK, the PCI switches you speak of
> > > are vaporware at the moment, while Fast Ethernet switches are not.
> > 
> > So, here I'm confused. This board that is being discussed
> > is using PCI as an interconnect, right? And the I2O
> > protocols are being used as the basis for a TCP/IP
> > implementation? Now, I know very little about I2O -- is
> > I2O only point-to-point? How would a broadcast work
> > on one of these eight-SA110 boards if PCI doesn't
> > allow for broadcast?
> 
> I know nothing.  I assume a broadcast would work by first sending to
> the first CPU, then the second, then the third . . .

Something like this. Alot depends on exactly how the meesaging is done but
typically if the data is held in shared memory you just need to keep
reference counts and send a message to each process in the network etc.

> > Both PCI switches and this board are currently vapor,
> > although it seems that an attempt to assemble these
> > SA-110 boards may be soon 
> 
> Yes, I think it should be next month that the first CPU cards are
> built, and the month after that that the main PCI boards go into
> production.

Sorry if I am repeating stuff that has already been mentioned ;-)

What switches are being talked about ? I have one just joined the list so
I missed the start of the thread.
A PCI switch is being looked at for the PCI card which will give each card
2 PCI busses which should help with communications as processors on the
same card will be able to communicate with each other whilst the host PCI
bus is being used for other communications etc.

> > and I haven't identified anyone
> > attempting to build a PCI switch, only a ring. Still,
> > there could easily be several iterations before all the
> > timing and software issues are worked out to the point that
> > the SA-110 boards can be stamped out and sold in volume;
> 
> This is conceivable; I don't know much about the issues involved here.

PCI bus switches do exist they just don't switch between many busses yet.

> > few new circuits of this complexity 
> 
> I think they are rather simple, actually; the CPU cards will have a
> CPU, some memory chips, and a PCI bridge circuit on them, and (I'm told
> thirdhand) that's it; the main PCI card will be even simpler.

The will also be some flash memory for boot firmware.

> > are fully functional
> > on the first try, I would suspect. Until that time, it
> > seems to me that in this thread we are discussing what
> > might be and not what is. Or have I missed something?
> 
> You are correct.  The SA-110 boards are relatively solid vapor, though
> -- two or three months, I think.  They're just a simple matter of
> engineering well-understood stuff.  I don't know about PCI switches and
> stuff; that sounds more like a simple matter of black magic.

Certainly the engineering is known, we have done other StrongARM boards
etc.
We don't want to go overboard with complexity to start with certainly you
could arrange PCI busses in hypercubes etc. with lots of fancy switching
but we are sticking to something fairly straight forward and something we
can engineer on a short timescale.

> The SODIMM-slotted Fast Ethernet adapters are also total vapor, though;
> nobody's started trying to design them, although I can't imagine it'd
> be hard -- they're just PCI Fast Ethernet cards with a funny form
> factor.
This is just a suggestion of something that could be done and something
that we have thought of. If there is a demand such things could appear.

Cheers,
				Mark