<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">Hi All,</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> The following are the requirements, which should be implememted through Sun Grid Engine:</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">1. Q1 - One queue with all the cores (e.g, 2 nodes, 8 cores)</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">2. Q2 & Q3 - 2 more queues under the large queue (6 cores & 2 cores)</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">3. user1 & user2 allowed to submit job to Q2</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">4. User3 & User4 allowed to submit job to Q3 only</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">5. Also User1 is allowed to submit job in Q1. (Brustable)</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">6. Can Application be only allowed to run through Q2?</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">7. Can Application be allowed to run through Q3 & Q1?</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">The cluster has 2 systems(1 master + 1 node). Each is Dual core, Dual Processor.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">So 4 cores each. </span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">Totally 8 cores.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">In the following I explain, what I've done and what not..</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">Created queues:</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">q1: Hosts=2 Master+Node slots=4</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">q2: Hosts=2 Master+Node slots=3</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">q3: Hosts=1 Node slots=1</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">Users:</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">user1</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">user2</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">user3</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">user4</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">Usersets:</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">userset12</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">userset34</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">Also,</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">user1 & user2 belongs to userset12</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">user3 & user4 belongs to userset34</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">1. Q1 - One queue with all the cores (e.g, 2 nodes, 8 cores)</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> </span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> Created queue q1, with 2 nodes, slots=4</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">2. Q2 & Q3 - 2 more queues under the large queue (6 cores & 2 cores)</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> </span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> Mentioned above.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> How to do that these two queues should fall under q1, the large queue?</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> May I know what are subordinate queues? </span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> If we make q2 and q3 subordinate to q1(q2 of 6 core & q3 of 2 core), does it meet our requirement?</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> If not, is it possible to do it in other way? </span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> </span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> </span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
3. user1 & user2 allowed to submit job to Q2</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> I've given access userset12 to q2. By this user1 and user2 can submit the jobs to q2.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> </span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> and given userset34 as xuserset.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">Now if user1 or user2 submit a parallel job of 6 mpi process, will it take 4 cores from master and 2 core from Node?</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">I tested it. 6 process job was not getting executed. But 3 process job got executed.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">The error is:</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
...</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
....</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
parallel environment: mpiq2 range: 6</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
scheduling info: has no permission for queue "<a href="mailto:all.q@compute-0-0.local">all.q@compute-0-0.local</a>"</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
cannot run in queue instance
"<a href="mailto:q1@compute-0-0.local">q1@compute-0-0.local</a>" because it is not contained in its hard queue
list (-q)</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
has no permission for queue "<a href="mailto:q3@compute-0-0.local">q3@compute-0-0.local</a>"</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
has no permission for queue "<a href="mailto:san.q@compute-0-0.local">san.q@compute-0-0.local</a>"</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
has no permission for queue "<a href="mailto:all.q@locuzcluster.local">all.q@locuzcluster.local</a>"</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
cannot run in queue instance
"<a href="mailto:q1@locuzcluster.local">q1@locuzcluster.local</a>" because it is not contained in its hard queue
list (-q)</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
has no permission for queue "<a href="mailto:san.q@locuzcluster.local">san.q@locuzcluster.local</a>"</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
cannot run in PE "mpiq2" because it only offers 0 slots</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">Means, it is not running when mpi processes are more than 3.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
May I know what went wrong here?</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">In case of serial jobs its working.If user1/user2 submits 6 serial jobs, 3 gets running on master
and three on compute node. If a 7th job is submitted, it will be queued
& waiting and starts to run when the slot becomes free.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">So the problem with parallel job has to be resolved.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> </span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">4. User3 & User4 allowed to submit job to Q3 only</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
Given access to userset34 and xuserset=userset12</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
If user3 or user4 submits a 2 process job, job gets submitted but doesn't execute. Error is:</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
parallel environment: mpiq34 range: 2</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
scheduling info: has no permission for host "locuzcluster.local"</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
has no permission for host "compute-0-0.local"</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
cannot run in PE "mpiq34" because it only offers 0 slots</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">The PE config is as follows:</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
$ qconf -sp mpiq34</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
pe_name mpiq34</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
slots 2</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
user_lists userset34</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
xuser_lists userset12</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
start_proc_args /share/apps/MPICH2/startmpi.sh -catch_rsh $pe_hostfile</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
stop_proc_args /share/apps/MPICH2/stopmpi.sh</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
allocation_rule $fill_up</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
control_slaves TRUE</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
job_is_first_task TRUE</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
urgency_slots min</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">I'm not getting how to resolve this issue. There might be something wrong in the settings.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">5. Also User1 is allowed to submit job in Q1. (Brustable)</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> For this I've added user1 to the owner's list and userset34 as xuserset.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">6. Can Application be only allowed to run through Q2?</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
7. Can Application be allowed to run through Q3 & Q1?</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"> Still these two has to be implemented.</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">Can anyone help me out to resolve above mentioned issues?</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;"><span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">Thanks,</span><br style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">
<span style="color: rgb(51, 102, 255); font-family: trebuchet ms,sans-serif;">Sangamesh</span><br>