[Beowulf] followup on 1000-node Caltech cluster

[Stuart Midgley]

Morning

An interesting system.

We have had issues with air-cooling racks that generate more than  
about 8kW of energy.  We just couldn't get enough cold air into the  
front of the rack.  We also had issues of air from the back of the  
rack recirculating around to the front of the racks (on the edges of  
the cluster)... so we had to build baffles to try and prevent this  
circulation.  I will be intersted to see how you go.

On 19/06/2005, at 7:04, David Kewley wrote:
>
> Our compute node racks will dissipate up to ~14kW.  The power  
> strips are
> three 5.7kW 208V 3-phase APC networked/switched "rack PDUs".  The
> intakes for the HVAC units are ~3 feet from the backs of the  
> racks.  We
> expect to have almost laminar air flow, but with good local & roomwide
> mixing due to an up&down airflow pattern from the supply ducts.


When we lost air conditioning to our machine room (we currently have  
about 300kW of gear) the room went up about 10 degrees celcius in  
about 10 minutes.  Now, we have automatic processes to shut machines  
down once they get to 40 degrees for a minute.  BUT the problem isn't  
the absolute temperature, its the rate of rise that kills gear.  Even  
though our machine hadn't shut down (it didn't reach the critical  
temperature) we were loosing disks and dimms all over the place just  
due to the sudden rate of rise of temperature.  When we spoke with HP  
and check out contract with them, it was quite clearly stated that  
the environment wasn't to go up more then 10 degrees in less than an  
hour.

Your machine sounds a lot larger than our old one... so I expect your  
problems to be worse.


> It was calculated that if the air cooling suddenly stopped while the
> power into the room was 400kW, but the fans kept the air circulating,
> the average rate of temperature increase would be 4 degrees F per
> second.  I remain a bit skeptical of that number personally -- for
> example, there was no allowance for the heat capacity of the
> now-stationary chilled water in the heat-transfer coils, nor of the
> walls, floors, ceiling, air ducts, and other surfaces in the room.   
> But
> the Planetary Science folks here know their atmostpheric modeling  
> (it's
> what they do for a living), so I'm pretty confident that their results
> are correct given their simple assumptions.


Good luck with it all
Stu.


--
Dr Stuart Midgley
sdm900 at gmail.com