[Beowulf] Please help to setup Beowulf
hahn at mcmaster.ca
Tue Feb 17 22:32:53 PST 2009
> searches. Without array task scheduling this would require 500,000 individual
> job submissions. The fact that I never met a serious PBS shop that had not
what's wrong with 500k job submissions? to me, the existence of "array jobs"
is an admission that the job/queueing system is inefficient. if you're
saying that the issue is not per-job overhead of submission, but rather
that jobs are too short, well, I think that's a user problem. I think it's
entirely reasonable to require user jobs to consume some minimum cpu time
(say, few minutes).
> - Policy and resource allocation features are very important to people
> deploying these systems
so I'm curious what that means. things like "dept A needs to be guaranteed
N cpus, but dept B gets to use whatever is left over"? or node choice
based on amount of free disk? I don't really see why these sorts of issues
would be less important to more parallel environments.
> - Storage speed is often more important than network speed or latency in many
which makes me wonder: do bio types consider using map-reduce-like
frameworks? that is, basically distributing the work to the data.
More information about the Beowulf