[Beowulf] Alternative to MPI ABI

[Jeff Squyres]

I have tried to reply to Greg's answer to my long-winded prior post 
about my thoughts on an MPI ABI 
(http://www.open-mpi.org/community/lists/users/2005/03/0021.php), but I 
simply have not had the time to compose a suitably-detailed/precise 
reply.

Instead, I would like to propose an alternative to an MPI ABI.

Create a new software project (preferably open source, preferably with 
an BSD-like license so that ISV's can incorporate this software into 
their products) that provides a compatibility layer for all the 
different MPI implementations out there.  Let's call it MorphMPI.  It 
would contain the following main components:

1. its own mpi.h / mpif.h
2. its own wrapper compilers (mpicc et al.)
3. its own library (perhaps named libmpi.*)

mpi.h contains all the normal mpi.h things (prototypes for all the MPI 
and PMPI functions, declarations of all the MPI constants, etc.), and 
then potentially a remapping from MPI functions to MorphMPI functions 
(e.g., "#define MPI_Send morph_mpi.mpi_send", where morph_mpi is a 
struct full of function pointers).

The wrapper compilers do the standard wrapper compiler things, enabling 
finding mpi.h / mpif.h, automatically finding and linking to MorphMPI's 
library(ies), etc.

The library is where the bulk of the work will be.  In MorphMPI's 
equivalent to MPI_INIT (perhaps named Morph_MPI_Init()), it dlopen's a 
back-end MPI implementation and sets oodles of internal tables to point 
to the back-end MPI functions and constants.  For example, 
morph_mpi.mpi_send is set equal to the result of a dlsym to find the 
symbol for "MPI_Send".

Morph_MPI_Init() can do some clever / user friendly things to pick 
which back-end MPI to dlopen, what dependent libraries also need to be 
dlopen'ed, etc.  This can be arbitrarily feature-ized.

There's still some technical issues to solve, but an industrious 
developer can figure them out.  For example, how to handle MPI 
compile-time constants (e.g., "MPI_Comm mycomm = MPI_COMM_WORLD;")?  
One possible solution is to have MorphMPI have a wrapper function for 
each MPI function (e.g., "#define MPI_Send Morph_MPI_Send").  The 
wrapper function does a translation of the MorphMPI MPI handles to the 
back-end MPI handles.  If MorphMPI's handles are integers, this can be 
relatively straightforward, something along the lines of:

int Morph_MPI_Send(...dtype, ..., comm) {
   return morph_mpi.mpi_send(..., Morph_MPI_datatypes[dtype], ..., 
Morph_MPI_communicators[comm]);
}

You get the idea.

There's a slight performance penalty for the translation layer, but for 
those who want an MPI ABI, this might well be an acceptable price to 
pay.

------

Of course, such a compatibility layer doesn't have to be *exactly* like 
this.  I simply proposed one possible implementation -- there's several 
other, similar ways to do it.  S/He who implements, wins.  :-)

The main ideas of this proposal are:

1. A 3rd party project can provide MPI ABI-like functionality (with all 
the benefits and drawbacks therein)

2. Cancel the MPI Implementor's Ultimate Prize Fighting Cage Match on 
pay-per-view (read: no need for time-consuming, potentially fruitless 
attempts to get MPI implementors to agree on anything)

3. With an appropriate FOSS license, anyone who wants ABI-like 
functionality can have it, but those who don't want it don't have it 
forced upon them

4. MPI implementors can keep doing what they do best: working on making 
their software great

This seems like a perfect project for a bright Master's student.  
Anyone care to open up a SourceForge project for it?  :-)

-- 
{+} Jeff Squyres
{+} The Open MPI Project
{+} http://www.open-mpi.org/