[Beowulf] 96 Processors Under Your Desktop

[Jim Lux]

----- Original Message -----
From: "Michael Will" <mwill at penguincomputing.com>

> There already is the opteron 246HE specced as 55W, that comes at twice the
> cost of the standard opteron 246 which is specced as 70W.
>
> AMD also announced a 246EE specced at 30W.
>
> What is the range of per year cost for a dual opteron 1U for air
conditioning and
> power consumption?
>
> Michael

Depends on where you are and what the incremental kWh rate is.

There was some discussion a while back on the list of what the electricity
cost to move heat is.. maybe 10-20% of the heat you're moving?  That is, it
takes 200W to move a kilowatt of dissipation out of the room. (guessing
here, and too lazy to recompute or search the archives, even though I
probably did the original post).

Probably more significant would be capital cost infrastructure issues,
triggered by moving over some threshold power dissipation.  For instance,
adding 1500W to my office dissipation would probably only cost about $400/yr
for electricity (assuming 8hrsdy,5dys/wk), but would trigger the
installation of additional a/c which would cost as much as the computer.
Putting it in an existing machine room, though, wouldn't likely trigger any
capital costs.

I doubt that anyone can cost justify using more lower powered processors
(i.e. fewer watts/GFLOPS) on a purely $/completed machine operation basis,
except in some fairly unusual environments (i.e. a spacecraft, where every
watt/joule is precious and expensive).

The real value (to my mind) is making a cluster a minimal-hassle item, the
same way a desktop PC is perceived today. Yes, I know that desktop PCs
require support, etc., but it's done on a mass production basis, much the
same as any other piece of office equipment, such as a copier, fax machine,
or telephone. desktop PCs at most large companies run pretty much the same
operating system , with efforts to have them all at the same revision, with
similar hardware configurations, etc.  Nobody obsessess about tweaking the
last 10% of performance out of them.  Nobody calls facilities when you
install a PC to verifiy power distribution, strengthen the floor, add sound
isolation, change the air conditioning.  Nope, the box shows up, you plug it
in, call net-ops to make the drop live, and away you go.

I'd venture to say that nobody installs clusters like this yet...
(obviously, Orion is aiming to).. No, you spend a few weeks or months
researching combinations of mobos and peripherals and ordering them, arrange
for packaging, arrange with facilities to make sure you have enough power
and floorspace for that rack or shelf, agonize about thermal kill switches,
etc., fiddle with various distributions, rebuild your software a couple
times, etc. (basically, read RGBs book and take everything to heart) I'd
guess that it is impossible to do the following today:

On Monday, you determine that you have a problem requiring the crunch of,
say, 5-10 processors, and you scrounge up a budget of, say, $10-20K to do
it.
On Friday, you are computing on your new cluster, having spent perhaps 4
hours of your time getting the cluster ordered and installed.

But this IS entirely possible with a desktop PC today. At work, I can go to
a website run by the institutional computing resources folks, pick from a
half dozen computers of various configurations, and have it installed and
running within a week.  It might cost me 2 hours to decide and place the
order, and another couple hours to sit with the installer while they drop it
off, hook it up, make sure it works, etc.  For this, I might pay
$300-500/month total, for my equipment cost, remote and desktop support,
etc.  The hardware cost is probably a few thousand dollars, but that's
essentially financed in the monthly tab.

Say everything scales with the hardware cost.  So, for that $10K-20K system,
I'd be paying $1K-2K/month (which fits right in with the traditional test
equipment rental cost strategy of 1/10th the purchase price per month (but
that includes calibration and maintenance and support)).  1-2K a month is
not a huge sum in the context of even a fairly small project, considering
that a full time body working on the "main problem" will probably set you
back $150-200K/yr. And, I don't need to buy and read any books about "how to
build and program a cluster", or go to conferences, subscribe to clustering
journals, spend time on the beowulf mailing list, etc.

Take an example from the RF test equipment area.... relatively few people
know how a vector network analyzer (a $50-100K piece of microwave test
equipment that measures 2 port parameters of a device) works inside, even
fewer know how to build one from scratch, but Agilent sells quite a few of
them to ready and willing customers, because it makes a job like tuning a
multisection filter a lot easier and faster than using the older alternative
(slotted lines, sweepers, etc.).  Sure, one COULD do it with the old
technique, but given the choice, you'd choose the VNA any day.  You'd have
to be in an unusual situation, today, though, to build your own VNA (two
situations: no money and lots of time, or, working at a frequency where
commercial VNAs or test fixtures aren't available)

Cluster computing is sort of where VNAs were a few decades ago.

Jim Lux