[Beowulf] 'liquid cooled' racks
ballen at gravity.phys.uwm.edu
Mon Dec 4 11:29:04 PST 2006
>> For my next cluster room, I am hoping to use 'liquid cooled' racks make by
>> Knurr (CoolTherm, http://www.thermalmanagement.de/). The scale is 67 racks
>> x 7.5kW heat removal per rack (36 U usable per rack).
> 7.5 KW/rack isn't much; are you designing low-power nodes?
Yup -- I guess so. For example our current cluster nodes are dual core
Opteron 175s (Supermicro H8SSL-i motherboards). They cost about $1200 for
2 x 2.2 GHz, with 1GB per core, and use 180 W under load (per 1U).
I will have enough rack space (67 x 34 U, per 500kW power and cooling)
that if I want to use nodes that dissipate 300 W per node then I will
simply limit myself to 25 nodes per rack.
>> The racks have front and back doors that close, and contains fans in the
>> back which circulate air through a heat exchanger located in the bottom of
>> the rack. The heat exchanger transfers the heat into chilled water.
> APC has something similar with the cooling on the side.
Can you give a positive or negative opinion about either the Knurr or APC
racks? Have you used them yourself in a system?
>> The advantages of this are that it is quiet, and that you don't need
>> vertical height for underfloor ducting or overhead hot air removal.
>> Disadvantages are cost, potential difficulty of working within the rack.
>> and loss of one rack of cooling capacity if you open both the front and
>> back doors.
> I'm pretty skeptical of the sealed-pod approach, since it seems to
> multiply the number of parts, create access issues, doesn't seem to
> actually save on space, etc.
It does save on vertical space, and it reduces noise to the level where
you can work comfortably in the room.
> I've also been burned by cold water cooling, so to speak (assuming you
> don't have your own, well-controlled CW plant.)
I'll have my own self-contained plant, designed by German engineers. They
seem to know their stuff.
> I would definitely consider a normal big-chillers approach _with_
> back-of-rack CW boosters (heat exchangers).
Can you provide a URL or recommendation for these back-of-rack CW
> and I'd definitely consider creative layouts of racks and chillers (for
> instance, I'm not crazy about the hot/cold-aisle approach - something
> W-shaped would be better for my ~50-rack machineroom. or even just
> lining all the hot racks up against the chillers along one wall.)
>> (In this case I have access to 'building' funds that can not be used to buy
>> more cpus, so the cost issue is not important.)
> are you sure you can't just be clever about laying out normal front-to-back
> racks? possibly with back-of-rack heat-exchangers?
I'd like a URL for the back-of-rack heat exchangers, please!
(PS: though that approach still sounds noisy!)
More information about the Beowulf