MPI/Beowulf vs. SM Programming/OpenMP

[Rusty Lusk]

Dear Tim,

| As I see it, with MPIch, I must have everything in a single program.

This is false.

MPICH can indeed be used for MPI programs in which the processes execute
different binaries.  This is not a new functionality; it was there in the very
first version, since it is inherited from p4, which also had that capability.
How to do it is clearly described in the User's Guide that accompanies MPICH.
Basically, one associates process ranks, hosts, and executable names in a file
and gives that file as an argument to mpirun.  Since MPICH has always
supported heterogeneous hosts, it has necessarily always supported multiple
executables.

MPICH can be configured to use a number of "devices" for its transport
mechanism.  The default device is the ch_p4 device, in which processes
communicate over sockets.  Many other devices can be configured, including
shared-memory and switches such as that used on the IBM SP.  Combinations
of shared memory and sockets are also possible.  

The restriction that you might have been referring to is that processes that
communicate through shared memory must (currently) be executing the same
binary.  Thus there could be an MPMD program in which several executables are
being used, but when processes with different executables communicate, they
use sockets.  LAM does not have this restriction.  So you are correct in the
sense that LAM has a capability that MPICH does not yet have, but it would be
quite misleading for anyone to state that MPMD programs are not supported by
MPICH.  If you hear anyone say this, please correct them.

Regards,
Rusty Lusk