[Beowulf] The Walmart Compute Node?
diep at xs4all.nl
Thu Nov 8 19:38:33 PST 2007
On Nov 9, 2007, at 1:27 AM, Chris Dagdigian wrote:
> It is dangerous to project *your* particular use cases and
> workflows upon the community at large.
> Most of the clusters I end up building or working on (academic,
> government and corporate sites) are intended to support periodic
> spikes in computing demands. For university sites this could be the
> end of each semester as student projects become due and for
> research labs it may be for a 10 day period proceeding the
> submission of a major paper or grant applications. For EDA
> companies entire clusters may lay idle until some massive
> validation process needs to kick off.
> Clusters built to meet peak demand rarely hit 90% utilization
> (averaged over time) and often have lots of idle capacity sitting
> around waiting for a peak period to arrive. That is why we pay
> particular attention to things like Project Hedeby from Sun and the
> EGO stuff from Platform Computing along with the various homegrown
> based systems that people have built to power-on and power-off
> nodes (via IPMI) based on the length of the pending job list
A few remarks:
a) the 70% usage comes from "supercomputer report europe".
b) "the community" that posts here has like NEAR TO ZERO $2500
clusters at their work,
so if you happen to know 1+ then that already gives statistical
significant bragging rights.
Ever seen a company say: "heh here you got $2500 build me the fastest
cluster you can get for that money"?
Actually i'm typing at an ex-company machine, a macbookpro 17'',
which i got from my previous employer and it is still 2600 euro in
the shop here, which soon is a dollar or 5000.
In general when crunching becomes important to a company, definitely
a billion euro company, then they're gonna invest quite some more
than $2500 into crunching power.
c) homegrown clusters usually are not built at the same manner like
the $2500 project says. Usually people buy a machine now, buy one a
year later and so on, and just cluster 'em, so instead of trying to
stick strict in some $2500 budget it's more like: "which cpu at what
cheapo mainboard gives most dang for my bucks". that'll be a quadcore.
d) for $2500 private clusters (private as in: at home) energy costs
play a big role.
e) another result of increased energy costs is of course sound concerns.
Regarding C: A far more interesting theoretic question is of course
whether you should see most homegrown clusters as a cluster or as
some jbon (just a bunch of cheapo nodes), as i can't really recall
most 'private house clusters' to have even MPI installed. When is it
heh you sure you still want your 2 cents in dollar currency rather
> In systems built for peak power and not constant throughput power
> control is a big deal and the eventual goal I'd love to see is more
> grid schedulers and resource brokers becoming hardware aware to the
> point of being able to power on and off nodes based on a given
> policy. It's coming and I've seen some neat homegrown solutions
> My $.02 as always
> On Nov 8, 2007, at 7:02 PM, Vincent Diepeveen wrote:
>> Building a $2500 cluster in order to NOT run software at it then
>> nonstop beats the idea of building a cluster. <snip> That assumes
>> you actually RUN software and that you have a lack of processing
>> power nonstop. So the machines are running all the time.
>> Additionally it's a private cluster, not some government type thing.
>> I tend ro remember the government model assumed in the end 70%
>> usage effectively of processing power. That's not real true for
>> private users of clusters. You really get far above 90% usage.
>> So you can argue the idle states do matter in the end for energy
>> costs, but you definitely can't assume it's idle majority of the
>> time. Building a $2500 cluster in order to then not let it run day
>> and night definitely is a thrown away $2500.
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit
More information about the Beowulf