[Beowulf] DC Power Dist. Yields 20%

[Jim Lux]

At 09:38 PM 8/10/2006, Mark Hahn wrote:
>>http://www.eweek.com/article2/0,1759,2000867,00.asp
>>
>>20% is a lot, both in terms of consumption and cooling capacity. I'm
>
>the article says "up to 15%" at the facility level - did you get 20%
>by figuring a savings at the rack level, too?
>
>afaikt, the article is based on an assumption that everything is powered 
>by an online UPS, and probably that node PSU's are low-performance (say,
>65%).  sometimes studies like this ignore fact that incoming power is 
>inherently AC (that is, only start looking at efficiency given DC supply.)

ALmost all power supplies start with something like a bridge rectifier into 
a capacitor filter giving 380VDC or so.   This has a terrible power factor 
at low loads (the current is not sinusoidal, but peaks at each half cycle), 
so there's also a bunch of stuff in front to fix harmonic content and/or 
power factor.

A "real" DC distribution system would distribute HV DC around (so as to 
reduce IR losses).  The 48V bus systems are closer to this. But, most of 
what I've seen is essentially repurposed 12VDC or 24VDC input DC/DC 
converters (e.g. half a UPS)


>I'd love to see DC gain more traction - PSU's are certainly one of the 
>flakier components in our systems, though per node (HP DL145G2), they only 
>contain 2 of 14 fans (or of 18 moving parts).  I don't know whether
>there's a reason to think many small AC-DC PSU's would be less efficient 
>than a couple really big ones (factoring in the cost and inefficiency of 
>DC power distribution).


But PSUs are notoriously price sensitive, and so, tend to run at the ragged 
edge.  reliability means overdesign, which costs money.. bigger magnetics, 
which means more copper (the cost of which has tripled in the last year), 
more silicon, bigger capacitors,...

What "costs" for PSUs is variable loads.  It's easy to get 90%+ efficiency 
(DC in to DC out) at fixed load.  The real challenge is to get good 
efficiency at, say, 20% load.


>I'd certainly be interested in a distribution system (whether AC or DC)
>that avoided so damn many plugs and sockets and breakers and PDUs.
>I guess I'm more enthused about servers becoming lower-powered, and also 
>quite interested in better ways to dissipate the heat than raised floors 
>and traditional chillers...
>
>>curious how long it would take before DC supplied racks become cost
>>effective.
>
>well, there's already a standard DC supply - to the motherboard.
>one impediment might be that it's got +12, -12, 5, 3.3 and probably
>a couple others.  if it were just a matter of providing lightly
>regulated 12V, life would probably be a lot simpler.

When you're working in the world of switching power supplies (as opposed to 
linear series regulators like the venerable 78xx series), "lightly 
regulated" input isn't as important as "very stable load current".  The 
challenge of designing a switcher that can handle fluctuations of 5:1 in 
output current on a microsecond or millisecond time scale is 
substantial.  All that nifty pipelining and cacheing means that the supply 
current drawn by your computer system can vary pretty substantially on a 
fairly short time scale (but not short enough that tiny, cheap bypass 
capacitors will hold the voltage up enough).  Consider something where 
you've got a tight loop that operates out of cache for 90% of the time, and 
hits outside memory the other 10%. The power drawn by the processor core is 
high during the 90%, and the power drawn by the core is low during the 10% 
(while it waits), but the power drawn by the I/O line drivers and the 
external memory goes up.

All very, very tricky.



>  and I'm not sure MB's would be much more complicated, either, since the 
> current
>main consumer, the CPU, already has a fairly flexible and high-power
>onboard dc-dc converter.  (I wonder how efficient it is, typically...)

12VDC power is horrible to deal with.. low enough so that currents (for, 
say, 200W) are high, so you need high current components and IR losses are 
significant (200W at 12VDC is around 17A. (30W for a 0.1 ohm loss).. now, 
gang up 10 200W mobos, and you're up to 170A.  Even small resistances (0.1 
ohms) can be significant power losers..That 0.1 ohm connector is going to 
cost you 3kW).  There's a reason that there's great pressure in the 
automotive world to go to 42 V DC supplies.

You, really, really want higher bus voltages to work from.. say, 100 VDC or 
greater.  There's some advantage in staying reasonably low (so that you 
don't have to buy kilovolt rated switching devices).  But, higher voltages 
(>50V) start bringing in a whole host of regulatory issues in equipment 
destined for the consumer market (and.. the whole Beowulf thing relies on 
the economy of scale with huge consumer equipment volumes, no?).





James Lux, P.E.
Spacecraft Radio Frequency Subsystems Group
Flight Communications Systems Section
Jet Propulsion Laboratory, Mail Stop 161-213
4800 Oak Grove Drive
Pasadena CA 91109
tel: (818)354-2075
fax: (818)393-6875