batch systems with job deps (afterok)

[Jakob Østergaard]

On Wed, Sep 26, 2001 at 11:03:10AM +0200, hanzl at noel.feld.cvut.cz wrote:
...
> Typically, previous invocation of 'make' would still run when I have
> the next steps ready. Typing "make" again at this moment would cause
> both copies to work on unfinished step (second make would not wait as
> it should). (Typically, I might finish the next step on friday and
> would like it to be invoked during the weekend, as soon as possible.)
> However there might be a simple solution (like one 'make' in loop,
> until there is nothing to do).
> 
> 
> I wonder why my needs seem to be uncommon - is it because you guys on
> the beowulf list
>  1) are not as lazy as I am and you have the whole program ready in time?

If you believe that one, I have this big tower in london that I can sell
you really cheap - it's got a clock on it too  ;)

>  2) you change data/parameters rather then programs?

I believe it is rare to change your program, and have the new "incarnation"
depend on results from an older incarnation (eg. obsolete or at least different
version).

In fact, I can't imagine how you manage to keep track of how you got which
results from which code, with a pipeline of various incremental versions of
your software depending on results from other versions.   But maybe that's just
me   :)

>  3) you have plenty of time and do not need to work in parallel with
>     your cluster?

Well, most scientists don't have a lot of work to do, and besides, you 
need time to spend all that money too...   Eh...    ;)

> 
> I do not beleive any of these, please tell me why you all are not
> calling for job dependencies in any job spooling system :)

I think (2) comes fairly close to at least some of the truth.

Of course you change your algorithms.  But I have never worked with
multiple revisions of code depending on results from other revisions.
And I have never heard of anyone else (except you) doing so to such
an extent.

-- 
................................................................
:   jakob at unthought.net   : And I see the elder races,         :
:.........................: putrid forms of man                :
:   Jakob Østergaard      : See him rise and claim the earth,  :
:        OZ9ABN           : his downfall is at hand.           :
:.........................:............{Konkhra}...............: