Beowulfs can compete with Supercomputers [was Beowulf: A theorical approach]

[Greg Lindahl]

> > Many supercomputer sites use IBM SP machines, which are
> > clusters, so there
> > isn't that much to prove.
>
> One question: from what I remember the nodes within an SP
> are moderate sized SMP in their own right.

Right. They used to be single CPUs, though.

> Mixed mode programming
> (threads within a node, MPI between nodes) was becoming popular
> as a way of increasing scalability (or rather avoiding the
> imediate penalties).

No. The fact is that IBM initially shipped those machines so that you HAD to
use mixed mode programming to use all the CPUs. Mixed mode hardly increases
scalability, unless your MPI is pretty bad at sending local messages, as
IBM's is.

You can run models like MM5 in either mode. MM5 is faster (on an SGI) as a
pure MPI program.

The emperor has no clothes.

> Have you guys had any experience with comparing the performance of
> similar codes on an SP versus a fully distributed cluster with similar
> performance and number of processors?

No -- there no Alpha slow enough for such a comparison ;-)

The MM5 results I keep on showing has an IBM SP line on it. It is slower per
cpu, and scales similarly. The scaling limitation on MM5 is mostly load
imbalance, not interconnect.

> Also, speaking of weather prediction, des anyone know of any recent
> advances in getting the shallow-water model to perform well on
> a cluster?  I was always under the impression that this was a big
> sticking point for some of the more complex models.

I may reveal my ignorance here, but:

Shallow water models are not cache friendly, so the usual problem is that
they run only as fast as main memory does. Vector machines mostly have
relatively good main memory systems, so there's a strike against non-vector
systems. Shallow water models are nicely MPI-friendly, but since they're
shallow, they often don't have very much data, which is a strike against
slower interconnects.

Most clusters have both strikes. They're cost effective, but the absolute
performance level may not be what you want.

-- greg