Beowulf in a Box

[Bob Drzyzgula]

On Sun, Sep 27, 1998 at 09:45:19AM -0500, Troy Benjegerdes wrote:
> On Sat, 26 Sep 1998, Kragen wrote:
> 
> Well, for one, there's a serious scalability problem.. Once you run out of
> PCI slots, you're finished. The nice thing about actual networking
> hardware is that you either get bigger a switch or add more switches
> connected via trunk lines with a small penalty in bandwidth and latency.

One can bridge PCIs together, obviously, and there is
work on PCI ring and star technologies to deal with this
scalability problem. http://www.sebringring.com is one
reference to people working on this.

In addition, I imagine (although I don't have specific
knowledge that this is true) that it should be possible
to build a PCI crossbar switch. If you can build a PCI
switch, and each CPU can have a dedicated PCI port, protocol
modifications along the lines of full-duplex Ethernet
may be possible. (???)

One further observation that I can make is that it is
becoming more common for Processors or their matched core
logic chipsets (e.g. the 21285 mentioned in the design in
question) to include basic PCI functionality. Every P5 &
P6 chipset since the Neptune (perhaps before?) has had a
built-in PCI interface. Sun's UltraSPARC IIi chips have PCI
on-board the processor. In order to get to Fast Ethernet,
one generally has to go through PCI first. If PCI can
be switched, then the extra step of the Ethernet chip
can be eliminated. I don't really understand (perhaps someone
can enlighten me) why a computer system can be more efficient
moving bits across a PCI-attached Ethernet card than it can
moving bits across the PCI bus itself.

One last thought is that a "PCI switch" would not
necessarily have to be PCI inside. Much like (but the other
way around) a 3Com CoreBuilder 7000, which is 100% ATM on
the backplane, but can have fast Ethernet interfaces on the
front panel. This is an ATM switch but one can attach Fast
Ethernet devices. Suppose that the PCI bus to "PCI switch""
interface just looked like a multi-port Ethernet adapter to
the system. If you had 64-bit, 66MHz PCI, the switch port
could look like four gigabit ethernet interfaces to an OS,
except that you wouldn't necessarily have to implement
any Ethernet PHY at all.

Just a few random, ill-informed thoughts...

--Bob

-- 
============================================================
Bob Drzyzgula                             It's not a problem
bob@drzyzgula.org                until something bad happens
============================================================