[Beowulf] scheduler policy design

[Brian D. Ropers-Huilman]

On 4/19/07, Tim Cutts <tjrc at sanger.ac.uk> wrote:
> On 19 Apr 2007, at 12:42 pm, Toon Knapen wrote:
>
> > Dear all,
> >
> > I am looking for articles and/or books on `scheduling policy design`.
>
> I don't know of any books; but my experience is that generally the
> simpler the configuration the better, especially as far as throughput
> is concerned.

Agreed. Also, as mentioned, you need to be very careful to define what
it is you want to optimize. It's fairly easy to have a very high
system utilization ( > 95% ) if you have a good job mix, but it's much
harder to have a very low queue time.

> Optimising for other things, such as turnaround time or some form of
> "fairness" in allocation of machine time, is much harder, and
> virtually impossible if your users refuse to make estimates of how
> much memory or runtime they actually require.

This last statement is very true. In my previous job I actually had a
grad student working on a system that would assess a "penalty factor"
in terms of how many SUs we would charge for a job if the user was
wildly off in their estimation of how much wall-time or memory they
would consume. This was to be an effort at educating our users. As
with all projects, getting the infrastructure right took most of the
time so we never did complete the work before I moved to my new job
here at MSI. I plan to revive this project here.

I'm open to comments from the list on this, too.

-- 
Brian D. Ropers-Huilman, Director
Systems Administration and Technical Operations
Supercomputing Institute                           <bropers at msi.umn.edu>
599 Walter Library                                   +1 612-626-5948 (V)
117 Pleasant Street S.E.                             +1 612-624-8861 (F)
University of Minnesota                               Twin Cities Campus
Minneapolis, MN 55455-0255                       http://www.msi.umn.edu/