[Beowulf] scheduler policy design

[Tim Cutts]

On 19 Apr 2007, at 12:42 pm, Toon Knapen wrote:

> Dear all,
>
> I am looking for articles and/or books on `scheduling policy design`.
> More precisely I'm looking for information on how to define  
> policies for
> queue systems (like SGE, PBS etc) to optimise throughput, efficiency,
> avoid conflicts, how to optimise short and long queue's etc.

I don't know of any books; but my experience is that generally the  
simpler the configuration the better, especially as far as throughput  
is concerned.

Optimising for throughput, at least with an embarrassingly parallel  
workload of serial jobs like we have here, is trivial; a single first- 
come-first-served queue is optimal, as long as the code is well  
written, and doesn't block too much on shared resources like file  
servers or databases.

Optimising for other things, such as turnaround time or some form of  
"fairness" in allocation of machine time, is much harder, and  
virtually impossible if your users refuse to make estimates of how  
much memory or runtime they actually require.

Most of the scheduling tweaks we put in to improve fairness and  
response time actually have a negative impact on throughput and  
efficiency.

Regards,

Tim