<div dir="ltr"><div dir="ltr"><div>Thinking about liquid cooling , and the ebuillient cooling, the main sources of heat on our current architecture servers are the CPU package and the voltage regulators. Then the DIMMs.</div><div>Concentrating on the CPU die package, it is engineered with a flat metal surface which is intended to have a thermal paste to transfer heat across to a flat metal heatsink.</div><div>Those heatsinks are finned to have air blown across them to transport the heat away.</div><div><br></div><div>In liquid immersion should we be looking at having a spiky surface on the CPU die packages and the voltage regulators?</div><div>Maybe we should spray the entire board with a 'flocking'' compound and give it a matt finish!</div><div>I am being semi-serious. I guess a lot of CFD simulation done regarding air cooling with fins.</div><div>How much work has gone into pointy surfaces on the die package, which would increase contact area of course and also act as nucleation points for bubbles?</div><div><br></div><div>One interesting experiment to do - assuming the flat areas of the CPU in an immersive system do not have (non thermal paste) heatsinks bolted on:</div><div>take two systems and roughen up the die package surfacewith sandpaper on one. Compare temperatures.</div><div><br></div><div>ps. I can't resist adding this. Sorry Stu . <a href="https://www.youtube.com/watch?v=kHnifVTSFEo">https://www.youtube.com/watch?v=kHnifVTSFEo</a></div><div>I guess Kenneth Williams is a typical vendor Site Engineer.</div><div>pps. the actress in the redress had her career ruined by this film - she ver got a serious role again after perfectly being typecast.</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Tue, 6 Nov 2018 at 22:57, Prentice Bisbal via Beowulf <<a href="mailto:beowulf@beowulf.org">beowulf@beowulf.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div class="m_4669408001860479086moz-cite-prefix">On 11/06/2018 02:03 PM, Lux, Jim (337K)
wrote:<br>
</div>
<blockquote type="cite">
<div class="m_4669408001860479086WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">True
enough.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Ebullient
cooling does have some challenges – you can form vapor
films, which are good insulators, but if you get the system
working right, nothing beats phase changes for a heat
transfer. </span></p>
</div>
</blockquote>
If I recall what I learned in my Transport Phenomena classes in
engineering school, you need a reasonably high temperature
difference to get a stable film like that. For that to happen,
radiant heat transfer needs to be the dominant heat transfer
mechanism, in the range of operation we are talking about, the
temperature difference isn't that great, and conduction is still the
dominant form of heat transfer. <br>
<br>
Here's an example of what 3M Novec ebullient cooling looks like. It
doesn't look like it's anywhere near the film boiling regime: <br>
<br>
<a class="m_4669408001860479086moz-txt-link-freetext" href="https://www.youtube.com/watch?v=CIbnl3Pj15w" target="_blank">https://www.youtube.com/watch?v=CIbnl3Pj15w</a><br>
<br>
--<br>
Prentice<br>
<br>
<blockquote type="cite">
<div class="m_4669408001860479086WordSection1"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u><u></u></span>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"></span>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<div>
<div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext">
Beowulf [<a class="m_4669408001860479086moz-txt-link-freetext" href="mailto:beowulf-bounces@beowulf.org" target="_blank">mailto:beowulf-bounces@beowulf.org</a>]
<b>On Behalf Of </b>Prentice Bisbal via Beowulf<br>
<b>Sent:</b> Tuesday, November 06, 2018 8:17 AM<br>
<b>To:</b> <a class="m_4669408001860479086moz-txt-link-abbreviated" href="mailto:beowulf@beowulf.org" target="_blank">beowulf@beowulf.org</a><br>
<b>Subject:</b> Re: [Beowulf] More about those
underwater data centers<u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal">. And serviceability is challenging. You
need to pull the "wet" boards out, or you need to connect
and disconnect fluid connectors, etc. If you're in an
environment where you can manage that (or are forced into it
by necessity), then you can do it.<u></u><u></u></p>
</blockquote>
<p>I think everyone on this list already knows I'm no fan of
mineral oil immersion (It just seems to messy to me. Sorry,
Stu), but immersion cooling with other liquids, such as 3M
Novec engineered fluid addresses a lot of your concerns. It as
a low boiling point, not much above room temperature, and it
was originally meant to be an electronic parts cleaner
(according to a 3M rep at the 3M booth at SC a few years ago,
so if you pull a component out of it, it dries very quickly
and should be immaculately clean.
<u></u><u></u></p>
<p>The low boiling point is an excellent feature for heat
transfer, too, since it boils from the heat of the processor
(ebullient cooling). This change of state absorbs a lot of
energy, making it very effective at transferring heat away
from the processor. The vapor can then rise and condense on a
heat exchanger with a chilled water heat exchanger, where it
again transfers a lot of heat through a change of state.
<u></u><u></u></p>
<pre>Prentice <u></u><u></u></pre>
<div>
<p class="MsoNormal">On 11/05/2018 06:30 PM, Stu Midgley
wrote:<u></u><u></u></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal">I refute both these claims.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">You DO want to run your boards
immersed in coolant. It works wonderfully well, is easy
to live with, servicing is easy... and saves you almost
1/2 your power bill.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">People are scared of immersion
cooling, but it isn't that difficult to live with. Some
things are harder but other things are way easier. In
total, it balances out.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Also, given the greater reliability
of components you get, you do less servicing.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">If you haven't lived with it, you
really have no idea what you are missing.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Serviceability is NOT challenging.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">You really do NOT want to run
boards immersed in coolant - yeah, there's folks doing
it at HPC scale<br>
<br>
Whatever the coolant, it leaks, it oozes, it gets
places you don't want it to go. And serviceability is
challenging. You need to pull the "wet" boards out, or
you need to connect and disconnect fluid connectors,
etc. If you're in an environment where you can manage
that (or are forced into it by necessity), then you
can do it.<u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal">-- <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">Dr Stuart Midgley<br>
<a href="mailto:sdm900@gmail.com" target="_blank">sdm900@gmail.com</a><u></u><u></u></p>
</div>
</div>
</div>
<p class="MsoNormal"><br>
<br>
<br>
<u></u><u></u></p>
<pre>_______________________________________________<u></u><u></u></pre>
<pre>Beowulf mailing list, <a href="mailto:Beowulf@beowulf.org" target="_blank">Beowulf@beowulf.org</a> sponsored by Penguin Computing<u></u><u></u></pre>
<pre>To change your subscription (digest mode or unsubscribe) visit <a href="http://www.beowulf.org/mailman/listinfo/beowulf" target="_blank">http://www.beowulf.org/mailman/listinfo/beowulf</a><u></u><u></u></pre>
</blockquote>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</blockquote>
<br>
</div>
_______________________________________________<br>
Beowulf mailing list, <a href="mailto:Beowulf@beowulf.org" target="_blank">Beowulf@beowulf.org</a> sponsored by Penguin Computing<br>
To change your subscription (digest mode or unsubscribe) visit <a href="http://www.beowulf.org/mailman/listinfo/beowulf" rel="noreferrer" target="_blank">http://www.beowulf.org/mailman/listinfo/beowulf</a><br>
</blockquote></div>