<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
Hi,<div><br><div><div>Am 22.02.2008 um 09:23 schrieb Sangamesh B:</div><br class="Apple-interchange-newline"><blockquote type="cite"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">Dear Reuti & members of beowulf,</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">I need to execute a parallel job thru grid engine.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">MPICH2 is installed with Process Manager:mpd.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">Added a parallel environment MPICH2 into SGE:</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">$ qconf -sp MPICH2</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">pe_name MPICH2</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">slots 999</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">user_lists NONE</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">xuser_lists NONE</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">start_proc_args /share/apps/MPICH2/startmpi.sh -catch_rsh $pe_hostfile</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">stop_proc_args /share/apps/MPICH2/stopmpi.sh</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">allocation_rule $pe_slots</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">control_slaves FALSE</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">job_is_first_task TRUE</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">urgency_slots min</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">Added this PE to the default queue: all.q.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">mpdboot is done. mpd's are running on two nodes.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">The script for submitting this job thru sge is:</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">$ cat subsamplempi.sh</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">#!/bin/bash</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">#$ -S /bin/bash</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">#$ -cwd</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">#$ -N Samplejob</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">#$ -q all.q</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">#$ -pe MPICH2 4</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">#$ -e ERR_$JOB_NAME.$JOB_ID</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">#$ -o OUT_$JOB_NAME.$JOB_ID</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">date</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">hostname</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">/opt/MPI_LIBS/MPICH2-GNU/bin/mpirun -np $NSLOTS -machinefile $TMP_DIR/machines ./samplempi</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">echo "Executed"</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">exit 0</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">The job is getting submitted, but not executing. The error and output file contain:</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">cat ERR_Samplejob.192</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">/usr/bin/env: python2.4: No such file or directory</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">$ cat OUT_Samplejob.192</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">-catch_rsh /opt/gridengine/default/spool/compute-0-0/active_jobs/192.1/pe_hostfile</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">compute-0-0</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">compute-0-0</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">compute-0-0</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">compute-0-0</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">Fri Feb 22 12:57:18 IST 2008</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">compute-0-0.local</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">Executed</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">So the problem is coming for python2.4.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">$ which python2.4</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">/opt/rocks/bin/python2.4</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">I googled this error. Then created a symbolic link:</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"># ln -sf /opt/rocks/bin/python2.4 /bin/python2.4</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">After this also same error is coming.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">I guess the problem might be different. i.e. gridengine might not getting the link to running mpd.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">And the procedure followed by me to configure PE might be wrong.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">So, I expect from you to clear my doubts and help me to resolve this error.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">1. Is the PE configuration of MPICH2 + grid engine right?</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"></blockquote><div><br class="webkit-block-placeholder"></div>if you want to integrate MPICH2 with MPD it's similar to a PVM setup. The daemons must be started in start_proc_args on every node with a dedicated port number per job. You don't say what your startmpi.sh is doing.<br><br><blockquote type="cite"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">2. Without Tight integration, is there a way to run a MPICh2(mpd) based job using gridengine?</span></blockquote><div><br class="webkit-block-placeholder"></div><div>Yes.</div><br><blockquote type="cite"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">3. In smpd-daemon based and daemonless MPICH2 tight integration, which one is better?</span></blockquote><div><br class="webkit-block-placeholder"></div><div>Depends: if you have just one mpirun per job which will run for days, I would go for the daemonless startup. But if you issue many mpirun calls in your jobscript which will just run for seconds I would go for the daemon based startup, as the mpirun will be distributed to the slaves faster.</div><br><blockquote type="cite"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">4. Can we do mvapich2 tight integration with SGE? Any differences with process managers wrt MVAPICH2?</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"></blockquote><div><br class="webkit-block-placeholder"></div><div>Maybe, if the startup is similar to standard MPICH2.</div><div><br class="webkit-block-placeholder"></div><div>-- Reuti</div><div><br class="webkit-block-placeholder"></div><br><blockquote type="cite"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">Thanks & Best Regards,</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> <span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">Sangamesh B</span><br></blockquote></div><br></div></body></html>