clusters v SMP

[Alan Scheinine]

At CRS4 we have an IBM SP and a cluster of PCs.  I've been told by
users of commercial fluid dynamic software that the version for the
IBM SP is much better than the version for a cluster of PCs.
Moreover, an IBM SP can handle a very large amount of memory.
On the other hand, many programs written in-house use MPI and the
shared memory of an SP is not relevant.  Though we have Myrinet on
the cluster of PCs, the in-house programs that I have seen use domain
decomposition in which the limit to scaling is the imbalance of domain
size, whereas the communication with Fast Ethernet is sufficient.
For such programs, a cluster of PCs is much more cost effective.
Also, upgrades of a cluster can be done more often than a large
computer such as an SP.  My point is, the choice of computer depends
of the type of program to be run.  If you have a wide range of types
of programs, a large parallel computer would be the best choice for
some programs whereas the cluster of PCs would be the best choice for
other programs.  In such a case, a mix of both types of hardware would
be the most cost effective with regard to the cost of the hardware.
On the other hand, you need to consider the personnel cost of
administering two systems.

With regard to a cluster of PCs, the experience of administering Linux
is also very useful for desktop machines.  Before Linux we had Unix
running on Sun, IBM R6K, SGI, and Hewlett Packard.  If I created a
program with graphics and threads and other utilities not part of
of the base operating system, it was hard to find a particular workstation
in which everything was installed.  In contrast, with Linux on a cluster
and on desktop computers, it is easier to have all tools on all machines.
Fastest Fourier Transform in the West, Open DX, Portland Group Compilers,
XML tools, etc. you can have them on both the cluster and the desktop
so software development can be done on the desktop and the same program
will then run on the cluster without changes.

Alan Scheinine  Email: scheinin at crs4.it