[Beowulf] Re: Cluster newbie, power recommendations (Geater at Home)

[Robert G. Brown]

On Tue, 21 Mar 2006, Ed Karns wrote:

> ... this is NOT! a power saving concept. The total power "consumed" will be 
> done by the total number of system motherboards, plus RAM, plus drives, plus 
> monitor(s) plus added accessories. In fact by "stressing" 8 power supplies to 
> run 16 motherboards (and generating the extra heat), your total electric bill 
> will actually go up, not down.
>
> Ed Karns
> FireWireStuff.com
> (I have personally built more than 1000 ISA systems, plus quite a few of my 
> own design.)

Dear Ed,

Have you measured this?  I was under the impression that large switching
power supplies were (or at least are likely to be) more efficient than
small ones, and that power supplies reached their maximum efficiency at
around 70-85% of their peak load (there are some online papers that seem
to support that that I googled up).  The average peak efficiency seems
to be in the ballpark of 60% to 80%, with relatively few supplies that
are better than that.  The no-load loss seems to be in the 20W range
pretty consistently; you'll likely never drop much below this.

However, you're quite right that overall, there isn't much to be gained
from screwing around with shared power, that you'll only gain something
if you really work to know what you are doing, and that there are
cheaper and better ways to get the same basic savings without sharing.

>From what I'm finding with Google, if you care about efficiency the most
important consideration is to pick a "good" power supply in the first
place.  There are "good" supplies out there that reportedly can run at
efficiencies of ~90% at load.  They have a small additional marginal
cost, but save you that good old $1/watt/year in a cluster that stays on
all the time, which can add up to $10-20/supply/year relative to a cheap
inefficient power supply -- call it a couple of hundred dollars for a 16
node cluster.  This might break even the first year but save you money
thereafter at this rate.

There is a nice article here:

   http://www.extremeoverclocking.com/articles/guides/Power_Supply_Guide_11.html

that is nearly everything you ever wanted to know about power supplies.
It emphasizes one last very important point that argues against sharing
unless you work hard and know what you are doing: don't assume that a
"350 watt" power supply can run three 100 watt motherboards, or that a
"450 watt" supply can run four of them.  The "power" rating of a supply
is a sum of the power that is deliverable on several voltage rails
(where draw on one rail can DROP power delivered to another if you start
drawing near the peak capacity), but this power is actually DRAWN in
typical operation from one or two rails, especially the 12 V rail.  You
might find that a 470 W power supply can only guarantee delivery of 200
W on the 12V rail -- instead of running four systems you might be lucky
to get it to run two.

Note well that I'm not arguing with your conclusion -- generally
speaking, mucking around with a really big power supply and special
wiring harness and so on is a job for the serious hobbyist or computing
enthusiast and not to be undertaken lightly, and could end up breaking
even to losing a bit relative to just using one GOOD power supply per
motherboard very easily.

For all of that, it is pretty clear that on the FAR side of things using
a big supply is a "good" idea.  One that Rackable takes to the bank --
they use a single very large DC power supply that is roughly 93%
efficient to power an entire rack of motherboards in their custom
racks/chassis.  For giant server farms, this can reduce power
consumption relative to MOST PC power supplies by perhaps 20% (they
claim 30%, of course:-) which adds up to quite a lot of money saved in
operational cost when you're powering tens to hundreds of kilowatts of
systems all of the time.  There are some IDC white papers to that effect
out there that go over all of this.

    rgb

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu