Archives


- Beowulf
- Beowulf Announce
- Scyld-users
- Beowulf on Debian

[Beowulf] Please help to setup Beowulf

Many of your questions may have already been answered in earlier discussions or in the FAQ. The search results page will indicate current discussions as well as past list serves, articles, and papers.

Search

Mark Hahn hahn at mcmaster.ca
Tue Feb 17 22:32:53 PST 2009


> searches. Without array task scheduling this would require 500,000 individual 
> job submissions. The fact that I never met a serious PBS shop that had not

what's wrong with 500k job submissions?  to me, the existence of "array jobs"
is an admission that the job/queueing system is inefficient.  if you're 
saying that the issue is not per-job overhead of submission, but rather 
that jobs are too short, well, I think that's a user problem.  I think it's 
entirely reasonable to require user jobs to consume some minimum cpu time
(say, few minutes).

> - Policy and resource allocation features are very important to people 
> deploying these systems

so I'm curious what that means.  things like "dept A needs to be guaranteed
N cpus, but dept B gets to use whatever is left over"?  or node choice 
based on amount of free disk?  I don't really see why these sorts of issues
would be less important to more parallel environments.

> - Storage speed is often more important than network speed or latency in many 
> cases

which makes me wonder: do bio types consider using map-reduce-like
frameworks?  that is, basically distributing the work to the data.



More information about the Beowulf mailing list