<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
David Kewley wrote:
<blockquote cite="mid200509290920.07695.kewley@gps.caltech.edu"
type="cite">..
<blockquote type="cite">
<pre wrap="">That sounds like positive progress..
Did they name a reason this is happening, or are they taking steps to
send someone down with a scope to see what is happening?
</pre>
</blockquote>
<pre wrap=""><!---->Yes, they're sending people out. Liebert engineers say they have basic
understandings of what's going wrong, and have some possible ways to work
around it or solve it. I'll not say more at this time, to give them time
to work it out.
</pre>
</blockquote>
Fair.<br>
<blockquote cite="mid200509290920.07695.kewley@gps.caltech.edu"
type="cite">
<pre wrap=""></pre>
..
<pre wrap="">I apologize for making a snide comment.
</pre>
</blockquote>
No problem, you are obviously under some stress and pressure on this,
so emotions do run a bit high.<br>
<blockquote cite="mid200509290920.07695.kewley@gps.caltech.edu"
type="cite">
<pre wrap=""></pre>
<blockquote type="cite">..
<pre wrap="">You have obviously decided, in advance, that the problem is with the
Liebert equipment.
</pre>
</blockquote>
<pre wrap=""><!---->
Maurice, in your two replies to this thread you've made lots of incorrect
inferences and assumptions, including this one. </pre>
</blockquote>
Perhaps so. <br>
I was basing what I said on what I read, however I have to apologize,
as I probably over-reacted, and have not heard the "whole story".<br>
<br>
OTOH, I am appalled by the fact that it has been "4 weeks since you
reported the problem to Dell and Liebert and apparently they have done
close to nothing about it.<br>
I know that on a job of this size, if you had bought from us, and
reported it, we would be all over it.<br>
Even Dell can not afford this kind of bad PR..<br>
<blockquote cite="mid200509290920.07695.kewley@gps.caltech.edu"
type="cite">
<pre wrap=""> Please show me the respect
of *not* assuming what I think, what I've done, or what others have done at
our site for this problem. If I've not *stated* some fact that you think
is important, simply ask me rather than assuming.
</pre>
</blockquote>
OK, fair comment.<br>
So, DID you get any useful response from either Dell or Liebert 4 weeks
ago, and in the interim.<br>
<blockquote cite="mid200509290920.07695.kewley@gps.caltech.edu"
type="cite">
<pre wrap=""></pre>
<blockquote type="cite">
<pre wrap="">You mention absolutely nothing about testing the power supplies.
That step should be the first, and fortunately is the easiest.
Almost any modern scope will do the job.
As it is low frequency it does not have to be an expensive or
specialized scope.
Instead of trusting "Kill-a-Watt clones" why not check the actual power
supply response, on a standard 115V single phase power input circuit?
</pre>
</blockquote>
<pre wrap=""><!---->
That's an excellent suggestion, and is in accord with my usual
troubleshooting & experimental inclinations. But because I have the
responsibility for *all* the aspects of commissioning this brand-new, large
cluster, I've had to leave lots of details to others, Liebert in this case.
</pre>
</blockquote>
So, delays are partially because the staffing at your site is short and
you simply do not have enough time to do what it takes to make it run?<br>
If so, I offer sympathy.<br>
I see this far too often. <br>
A budget of a million dollars for a cluster, but no cash to implement
it or maintain it.<br>
That must be very frustrating!<br>
<blockquote cite="mid200509290920.07695.kewley@gps.caltech.edu"
type="cite">
<pre wrap="">To the best of my knowledge, Liebert has not studied these exact power
supplies, but they say they understand PSes that are similar enough that
they can work out a model of our specific problem. Until I have time to
run experiments myself, I am going to trust them to cover these bases.
</pre>
</blockquote>
I would, in my experience they have a heck of a good rep.<br>
<br>
<blockquote cite="mid200509290920.07695.kewley@gps.caltech.edu"
type="cite">
<pre wrap=""></pre>
<blockquote type="cite">
<pre wrap="">I have seen power regulation equipment fail in a similar fashion before,
where the power supplies are pulling down too much current to the
neutral phase,
and making the power feed overload on one phase, driving it into
instability.
This is a classic symptom of cheap, poorly designed and made power
supplies. Or bad room wiring, with undersized neutral lines.
</pre>
</blockquote>
<pre wrap=""><!---->
The PDUs have a front panel that displays lots of diagnostic measurements,
and they sound a rather piercing alarm when any measurement goes over its
Liebert-defined limit (they are the only alarms I've heard in that room
that can reliably be heard over the room noise, from any part of the
room :). The PDUs also have suitably sized breakers and suitably sized
conductors on each of the 93 branch circuits.
The three output phase currents all stay well under their limits, even when
they begin to become unstable (at the low-power end of the instability, and
well into the instability domain). Toward the high-power end of the
instability domain that we've tested, the current oscillations become large
enough, and sit on top of a large enough average current, so the PDUs *do*
give overcurrent alarms (plus other alarms due to the wild oscillations).
Unless something is going on that is not alarmed for, the PDUs and the Liert
techs who've been onsite don't indicate any problem with the neutral wiring
or the power supplies per se.
</pre>
</blockquote>
So, what DO they think is causing this? I am really curious..<br>
<blockquote cite="mid200509290920.07695.kewley@gps.caltech.edu"
type="cite">
<pre wrap=""></pre>
<blockquote type="cite">
<pre wrap="">Liebert make big UPS and power units, and those are their "bread &
butter"
Frankly I am surprised they have not yet dispatched a tech down to your
site with test equipment by now..
</pre>
</blockquote>
<pre wrap=""><!---->
When did I say they haven't dispatched a tech to our site? In fact they
have, mutliple times; I just hadn't mentioned that up to this point in this
thread. </pre>
</blockquote>
Ah.. that paints one very different picture.<br>
So bascially Liebert are on it, you have not mentioned what, if
anything Dell have done, but your are coming to this list because after
some weeks you still are not seeing a solution happening?<br>
<br>
<blockquote cite="mid200509290920.07695.kewley@gps.caltech.edu"
type="cite">
<pre wrap="">My concern was not that they aren't sending techs, but that they
have no solution yet, and that I wasn't getting a warm-fuzzy feeling that
they really were treating this problem as critically as we need them to.
</pre>
</blockquote>
OK, so that tells me more..<br>
Have they identified what peice of equipment that they think is causing
the problem yet?<br>
<blockquote cite="mid200509290920.07695.kewley@gps.caltech.edu"
type="cite">
<pre wrap="">After yesterday's conference call, I feel better about their efforts. Even
so, the proof is still in the outcome, and the outcome is far from certain.
</pre>
</blockquote>
No kidding.<br>
<blockquote cite="mid200509290920.07695.kewley@gps.caltech.edu"
type="cite">
<pre wrap=""></pre>
<blockquote type="cite">
<pre wrap="">When you say "Liebert has been on this case for something like 4 weeks
now." what does that mean?
</pre>
</blockquote>
<pre wrap=""><!---->
That's when we first demonstrated this problem to their onsite tech &
engaged their help in solving it.
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">Can anyone here offer ideas, or better yet, experience?
</pre>
</blockquote>
<pre wrap="">I was trying to.
Apparently you do not appreciate suggestions, except ones that support
your distrust of Liebert.
</pre>
</blockquote>
<pre wrap=""><!---->
I appreciate all constructive suggestions. My appreciation does not extend
to insinuations.
Thanks for trying to offer ideas & experience. I *do* appreciate some of
what you've written in this email. I appreciate *none* of what you wrote
in your first reply to this thread -- if you like, go back and read it and
see if you can understand why.
</pre>
</blockquote>
OK, I apologize.<br>
Some of it is still becasue of what I will call a "selective
application of information".<br>
If you had mentioned things like what you say about Liebert's actions
to date, as you have in in this message, it would have painted a
different story entirely.<br>
<br>
<blockquote cite="mid200509290920.07695.kewley@gps.caltech.edu"
type="cite">
<pre wrap=""></pre>
<blockquote type="cite">
<pre wrap="">Why not test the power supplies?
If doing it yourself is not something you are comfortable with, there
are many electrical inspection labs in your region that provide this
service, usually for under $150.
Look in the yellow pages under "testing" or similar.
Many will allow you to stand there and watch and ask questions as they
do it.
</pre>
</blockquote>
<pre wrap=""><!---->
Now *that* is a very good suggestion. Thank you. I did not know testing
could be this easy. (By the way, I'm comfortable with testing / measuring
the power supplies, although I don't have the equipment on hand to do it
properly, and I don't have the full range of knowledge to interpret all of
what I measure.)
</pre>
</blockquote>
We have to do it regularly for custom equipment.<br>
To meet CSA, CE, and UL one gets what is called a "site inspection"<br>
Often the best and cheapest way is to take the piece to a certified
test labs and they do the test, provide a short report, and a sticker
certifying it is electricall safe and accceeptable<br>
It is not an FCC radio emissions test and certification, but you can
ask for that too, albeit at a higher cost.<br>
They measure power characteristics, PF, current leakage, consumption,
stability, load maximum, etc.<br>
<br>
<br>
<blockquote cite="mid200509290920.07695.kewley@gps.caltech.edu"
type="cite">
<pre wrap="">For now, I'm going to continue to let Liebert run with this problem; we've
offered to get them a power supply to take apart and/or measure, but so far
they seem to believe they understand it well enough. I'm also going to
trust Dell, that their power supplies are of good quality, just of poor
interaction with the rest of our power infrastructure.
Meanwhile, I have several other things to take care of on the cluster,
before users can get more than minimal use out of it, so I'm not yet going
to get into detailed measurements myself.
David
</pre>
</blockquote>
Good luck!<br>
--<br>
<br>
With our best regards,<br>
<br>
Maurice W. Hilarius Telephone: 01-780-456-9771<br>
Hard Data Ltd. FAX: 01-780-456-9772<br>
11060 - 166 Avenue <a class="moz-txt-link-freetext" href="email:maurice@harddata.com">email:maurice@harddata.com</a><br>
Edmonton, AB, Canada <a class="moz-txt-link-freetext" href="http://www.harddata.com/">http://www.harddata.com/</a><br>
T5X 1Y3<br>
<br>
</body>
</html>