Beowulf in a Box

[Troy Benjegerdes]

On Sat, 26 Sep 1998, Kragen wrote:

> On Sat, 26 Sep 1998, Kragen wrote:
> > http://www.pobox.com/~kragen/sa-beowulf/
> 
[snip]

> I'm interested to hear other people's comments.  William Rankin
> commented that such things have never gone anywhere, and I'm curious to
> find out why.7

Well, for one, there's a serious scalability problem.. Once you run out of
PCI slots, you're finished. The nice thing about actual networking
hardware is that you either get bigger a switch or add more switches
connected via trunk lines with a small penalty in bandwidth and latency.

I also think that without a native PCI-based MPI or other messaging layer,
you're going to get killed by TCP/IP overhead. Gigabit ethernet (and
myrinet) both only get around ~150 Mbits/sec TCP thoughput on a 200 Mhz
Pentium Pro. (UDP is better, around 200Mbits/sec or so, but still only a
5th of Gigabit ethernet wire speed).

There is also the problem of distributing computations to minimize
communication latencies (if this is even possible for the problem you're
dealing with). The aggregate bandwidth is great when transferring among
procesors on the same card, but not when all 8 procs on a card need to
talk to another board. 

All of these problems could potentially be dealt with in software, but I
suspect the development of this software will take on the order of 5 or 10
years, even as an open source project. Some of the MOSIX load balancing
algorithms might be quite usefull as a starting point. (I believe there
are papers on most of them floating around)


--------------------------------------------------------------------------
| Troy Benjegerdes    |    troybenj@iastate.edu    |    hozer@drgw.net   |
|    Unix is user friendly... You just have to be friendly to it first.  |
| This message composed with 100% free software.    http://www.linux.org |
--------------------------------------------------------------------------