[Beowulf] Do these SGE features exist in Torque?

[Craig Tierney]

Reuti wrote:
> Hi,
> 
> Am 09.05.2008 um 20:26 schrieb Prentice Bisbal:
> 
>> At a previous job, I installed SGE for our cluster. At my current job
>> Torque is the queuing system of choice. I'm very familar with SGE, but
>> only have a cursory knowledge of Torque (installed it for evaluation,
>> and that's it). We're about to purchase a new cluster. I'd have to make
>> a good argument for using SGE over Torque. I was wondering if the
>> following SGE features exist in Torque:
>>
>> 1. Interactive shells managed by queuing system
>> 2. Counting licenses in use (done using a contributed shell script in 
>> SGE)
>> 3. Separation of roles between submit hosts, execution hosts, and
>> administration hosts
>> 4. Certificate-based security.
>>
>> Are there any notable features available in Torque that aren't available
>> in SGE?
> 
> what you can find in Torque but not in SGE: request a mixture of nodes, 
> i.e. one heavy node with much memory (or big I/O options) and 5 nodes 
> with less memory or less disk performance for a parallel job.
> 

Not true.  Torque syntax is much cleaner for doing this, but you
can do it in SGE.  Call the big memory hosts "bigmemN", where N
is an integer from zero to how many ever servers you have.

Create a hostgroup with the compute nodes called "@compx".  Subsequently
create at parallel environment and queue that use this host group.
Normal jobs will go into this parallel environment.

Next, create a parallel environment called pebigmem.  Then,
for each host, create an individual queue called qbigmemN.q
(replace N with the appropriate integer).  For each queue,
specify the host list as:

hostlist   bigmemN @compx

When you launch a job, the call to qsub should include:

qsub -pe pebigmem 6 -masterq qbigmem0.q at bigmem0,qbigmem1.q at bigmem1,.....

The -masterq line should list every bigmem queue instance.

At our site, we have a qsub wrapper script that when a user asks
for the parallel environment "pebigmem", we add the -masterq line
to hide the details from them.

If I hadn't figured out how to do this 6 years ago, we could
have never migrated from OpenPBS to SGE.

Craig



> OTOH, if you have parallel jobs: 
> http://www.beowulf.org/archive/2007-September/019269.html
> 
> What is different between them from the idea: in Torque you submit a job 
> into a queue, while in SGE you request resources and SGE will select an 
> appropriate queue for you.
> 
> -- Reuti
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit 
> http://www.beowulf.org/mailman/listinfo/beowulf
> 


-- 
Craig Tierney (craig.tierney at noaa.gov)