Need ball park power and cooling requirements

[Ray Muno]

On Tue, Oct 01, 2002 at 03:12:52PM -0400, Robert G. Brown wrote:
> On Tue, 1 Oct 2002, Dan Sturtevant wrote:
> 
> > 
> > > I'm setting up lab space and plan to build a small (8-16 node) cluster
> > > of dual Xeon nodes (Dell 2650 or equivalent). I need to give the
> > > engineers the power and cooling needed in the 6'x8' server room that
> > > will house the cluster. Any rules-of-thumb for what's required? Thanks.
> > 
> > I would put each set of 8 nodes on a separate 20 amp circuit.
> 
> Also, I'd strongly suggest that you ensure that you are using power
> factor correcting power supplies in the nodes themselves AND make sure
> that your electrical contractors are competent.  For example, if they
> run multiple circuits with different phases, do not permit them to share
> a common neutral.  There are companies that make harmonic mediating
> transformers that can compensate for the harmonic distortion caused by
> switching power supplies, if it turns out to be impossible to get PFC
> supplies from Dell or whoever.
> 
> Because switching power supplies draw basically all of their current in
> the middle third of each voltage half-cycle: a) cancellation that might
> have occurred in shared neutrals becomes addition instead, leading to
> dangerous overheating of the neutral line; b) just using RMS power
> consumption and the assumption of voltage and currents in phase
> significantly underestimates the peak currents drawn by the systems --
> you may find your 20A breakers blowing when you are only drawing
> 1100-1200 watts instead of the 1500-1600 you might have expected to run
> on a 2400 VA line; c) PFC supplies consume less energy on average, and
> the power companies often charge you less money for what you use if you
> don't need the high peak currents associated with a relatively poor
> power factor.
> 

These are very good points.  We just installed a couple of 36 kVA UPS's
in our cluster room and main server room.  We had to call the electricians
back because the UPS manufacturer requires a neutral of double the ampacity
of the 3-phase primaries, either a pair of the same size as the primaries
or a single feeder that meets the rating.   The neutral current, for typical
data center loads, is rated at 175% of the primary phases.  

In contrast, we are housed in an old building (1948 era). All of the electrical
distribution panels (3 phase) have neutrals that are smaller then the 
primaries.  The main loads in the building when it was designed, were lights.
This causes us a great deal of problem now that the main loads are air
conditioners, computers, printers and copiers.  The harmonics present 
cause all kinds of problems with our lab instrumentation in the building.
To address this issue, we ended up bringing in a new 700 kVA 480V service
in to the building which meets "modern" requirements.  We were lucky to 
be able to tie our power for our cluster room to that instead of the existing
electrical service.  We are also in the process of gutting and re-feeding
the existing panels mainly to increase the neutral capacity. 

As many people have pointed out in the past, the facilities costs, many
time overlooked in the planning stage, are a significant share of the cost
of housing a large installation of computers.  


=============================================================================
 
 Ray Muno                           http://www.aem.umn.edu/people/staff/muno
 University of Minnesota                          e-mail:   muno at aem.umn.edu
 Aerospace Engineering and Mechanics               Phone:     (612) 625-9531
 110 Union St. S.E.		                     FAX:     (612) 626-1558
 Minneapolis, Mn 55455			

=============================================================================