[Beowulf] hybrid (openmp+mpi) job submit

[Peter Kjellström]

Hello,

There are quite a few ways to set the environment for remote mpi-processes in 
batch-jobs but they are often ugly or specific (or both). The root problem 
here is that simple batch-systems simply runs your batch script on the first 
node and that's it. Then mpirun is responsible for starting the rest of the 
the processes (often using ssh or, horrible thought, rsh..).

Here are a few ways to make it work (setting the enviroment variable that is, 
not actually running a hybrid mpi/openmp app ;-).

* try setting it hard in you .bashrc if you use bash (ugly)

* use LSFs tight mpi-integration (specific to LSF)

* use an MPI with an mpirun that propagates the environment


-- tosses .02 swedish kronor into the thread


On Friday 08 July 2005 22.49, turuncu at be.itu.edu.tr wrote:
> hi,
>
> i try to run a job that is parallelized using openmp and mpi programming
> interfaces (hybrid). I need to run mpi jobs in each node as an openmp job.
> for this reason, i have to define OMP_NUM_THREADS environment variable for
> each one of the node. first i try to put it into .profile file but it is
> not sucessful. also i try to write an LSF job script and i fail too. The
> LSF script as fallows,
>
> #!/bin/ksh
> #BSUB -J MM5_RUN         # job name
> #BSUB -n 2               # sum of number of tasks
> #BSUB -R "span[ptile=1]" # number of processes per node
> #BSUB -m "cn07 cn08"     # run host
> #BSUB -o mm5lsf.out      # output file name
> #BSUB -q cigq            # queue name
> #BSUB -L /bin/bash            #
> #BSUB -E "export OMP_NUM_THREADS=2"
>
> . ${PWD}/mm5.deck.par
> time mpirun -np 2 -machinefile ../machfile ./mm5.mpp
>
> in this case. job run in each of the specified node as a single processor
> mode (except execution host, because it is same machine which is login in
> and OMP_NUM_THREADS environment variable comes from .profile file).
>
> how can i run a command (or script) in each node just before runing mpi
> executable?
>
> thanks,
>
> Ufuk Utku Turuncoglu
> Istanbul Technical University
> Informatics Institute
>

-- 
------------------------------------------------------------
  Peter Kjellström               |
  National Supercomputer Centre  |
  Sweden                         | http://www.nsc.liu.se
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://www.scyld.com/pipermail/beowulf/attachments/20050711/9a5e226e/attachment.bin