[Beowulf] Intel MPI 2.0 mpdboot and large clusters, slow tostart up, sometimes not at all

[M J Harvey]

Hi,
> If you have a batch system that can start the MPDs, you should 
> consider starting the MPI processes directly with the batch system and 
> providing a separate service to provide the startup information.
You're exactly right. Intel's MPI is derived from MPICH2 and (as we use 
PBSPro)  OSC's mpiexec should do that job nicely, starting the MPI 
processes via PBs's TM API and then speaking to them via PMI. However, 
since version 2.0.1 refresh 1, Intel have used a modified (and 
incompatible) PMI command set for which documentation hasn't been 
forthcoming. Lacking the time to hack about, we've had to revert to 
using their mpd for the time being.

Matt

-- 
Matt Harvey                     Email: m.j.harvey*imperial.ac.uk
HPC Systems Support Analyst     Tel  : +44 (0) 20 759 47233
Imperial College London         Mob  : +44 (0) 77 251 59691

http://www.imperial.ac.uk/ict/services/highperformancecomputing