[Beowulf] scheduler policy design

[Tim Cutts]

On 20 Apr 2007, at 8:10 am, John Hearns wrote:

> Toon Knapen wrote:
>
>
>> Additionally, in the case above, for optimising the efficiency of  
>> the node, I might prefer to launch just 1 process which uses 4  
>> threads to perform multi-threaded (BLAS) calculations.
>
> We help out customers deal with problems like that, and the above  
> example on serial jobs.
> The answer to this one would be to create a parallel queue with  
> just one job slot, and prevent any other jobs from running on the  
> node. The parallel job then gets the full resources of the node,  
> and can run the multi-threaded calculations you want.
> If you don't want jobs to be allocated that way, we have a  
> 'multiway' queue for running several jobs per host.

But remember you don't necessarily need to have separate queues if  
your scheduler supports asking for multiple job slots on a single  
machine (as in the LSF example I gave in my last posting).  John's  
suggestion is an alternative way to achieve the same thing, although  
it introduces other questions, such as what's the relative priority  
of the two queues, and does the scheduler support backfill scheduling  
between the queues to help short single threaded jobs run while it is  
reserving processors for the parallel job?

Tim