[Beowulf] hybrid (openmp+mpi) job submit

[turuncu at be.itu.edu.tr]

hi,

i try to run a job that is parallelized using openmp and mpi programming
interfaces (hybrid). I need to run mpi jobs in each node as an openmp job.
for this reason, i have to define OMP_NUM_THREADS environment variable for
each one of the node. first i try to put it into .profile file but it is
not sucessful. also i try to write an LSF job script and i fail too. The
LSF script as fallows,

#!/bin/ksh
#BSUB -J MM5_RUN         # job name
#BSUB -n 2               # sum of number of tasks
#BSUB -R "span[ptile=1]" # number of processes per node
#BSUB -m "cn07 cn08"     # run host
#BSUB -o mm5lsf.out      # output file name
#BSUB -q cigq            # queue name
#BSUB -L /bin/bash            #
#BSUB -E "export OMP_NUM_THREADS=2"

. ${PWD}/mm5.deck.par
time mpirun -np 2 -machinefile ../machfile ./mm5.mpp

in this case. job run in each of the specified node as a single processor
mode (except execution host, because it is same machine which is login in
and OMP_NUM_THREADS environment variable comes from .profile file).

how can i run a command (or script) in each node just before runing mpi
executable?

thanks,

Ufuk Utku Turuncoglu
Istanbul Technical University
Informatics Institute