[Beowulf] launching python in parallel
toon.knapen at fft.be
Thu Apr 19 02:57:44 PDT 2007
I have been looking into launching python in parallel. There are
multiple tools currently doing this like PyTrilinos, PyPar, mpi4py etc.
However I was wondering if launching a standard python interpreter using
mpirun (or any mpi launch script) is portable ?
For instance, I have my small python module (let's say 'start.py') which
on its turn loads an extension module (written in C) which calls
MPI_Init. I launch my program using following cl:
mpirun -np 2 /my/path/to/python start.py
where start.py looks like
my_extension_module.init() # this will call MPI_Init
When I do this using mpichgm, it works. However using mpich-220.127.116.11
(ch_p4) it does not work. To me the reason is that ch_p4 passes the
cl-arguments in the MPI_Init. So whenever the remote python's are
launched, they are not told which script (start.py) they need to launch.
So therefore I create a short shell script (called start.sh) which
contains the following
However this does not work either. mpirun.ch_p4 refuses to launch the
second process (start.sh) Tracing the MPI_Init call made by the master,
he does not even try to create the other process (because he does not
get into the net_create_slave function).
So the question now is: is `mpirun -np 2 /my/path/to/python start.py`
supposed to work or not ?
Thanks in advance for any enlightenment on the subject !
More information about the Beowulf