Beowulf - Single Board Computers? (long)

[Bob Drzyzgula]

On Sun, Sep 20, 1998 at 10:41:40PM -0400, Greg Lindahl wrote:

> OIC -- I hadn't realized you had something so aggressive in
> mind. That's a pretty tough standard to meet, because your ability to
> have any kind of disk or expansion networking cards or anything else
> is very limited.

Yes. :-)

> www.altatech.com -- Alpha/Myrinet CompactPCI
> www.mc.com
> www.sky.com -- PowerPC, custom switched network

I'll look some. Sky I am familiar with. As you have pointed
out, however, per-node cost is a major issue in a Beowulf;
Sky's prices are... well, I won't say it. The idea of a
custom board has occured to me, but I can't imagine that
it would be cost effective in quantites less than a
thousand or so. Perhaps if a *lot* of people wanted to do this...

> > LAN-on-PCI may not be the cat's meow for current PCIs,
> > but it seems to me that the future holds some promise,

> Well, my crystal ball isn't any better than anyone else's, but
> ethernet switches are lightyears ahead of PCI busses, and are likely
> much higher volume. Yes, PCI interface silicon is high volume, but
> 99.99% of them are going to go into desktops which are happy with a
> single bus. Still, we should keep in mind that the only thing that
> makes Myrinet expensive is the interface cards, not the switches...

There've been some recent stories in Computer Design
magazine about alternative PCI architectures, but I'm
still digging in my archives (I use the "chronological
filing system" for paper :-) for the issues of interest so
I can give a reference. Here's a couple, but there was a
longer review of the Sebring ring and a more recent survey
article that I couldn't find on the website... they've
been supposedly been doing a major redisign for months.

http://www.computer-design.com/Editorial/1997/12/departments/1297DTPC.HTM
http://www.computer-design.com/Editorial/1997/07/networks/797netswitch.html

Sebring has a website, http://www.sebringring.com/

> 
> I'm not sure that AltaTech's commodity motherboard gizmo actually uses
> that many power supplies. The thing which makes it big is the disk and
> the ability to plug in a PCI card. Presumably if you had a motherboard
> with no disk and didn't need any cards because you were using built-in
> ethernet, you could get them to space the shelves closer than usual,
> giving you about double their current density of 32 systems per rack.
> 
> And it would be cheap. And it's only twice as big as your idea system.

This certainly would be worth looking at.

> Being as naieve about power as I am, would this beast use considerably
> less power if you had a consolidated gizmo for dishing out DC to each
> motherboard? My existing alphas with their 300? or 400? watt power
> supplies actually pull only 1 amp of 110V with just the cpu in use and
> 2 amps when they are using their disk drives. We measured this at the
> breaker box using one of those induction gizmos. If getting rid of the
> separate power supplies and not having a disk gets us down to 0.5 amps
> per CPU, we're getting close to your ideal number with commodity
> parts.

An ideal power supply should not consume any power, only
transform it into something that the device can use. In
reality, the power supplies in PCs do consume some power,
but they are relatively efficient. On the whole, suppose
that you had two pairs of systems, each connected back to
a single power strip plugged into separate circuits. Both
pairs of systems are configured exactly the same, except
one uses a dedicated power supply per system, and the
other uses one power supply with two feeds, then the
current measured at the wall outlets should be pretty much
the same.

The manual for the DK440LX says that a single-processor,
266MHz system running Windows 95 with a hard disk, floppy,
CD-ROM, video card and 64MB of memory should consume
about 50W avg. (measured at the AC outlet) without APM
enabled. Intel doesn't say, but I would imagine that a
heavily-worked Beowulf node would burn more. A second
processor, more memory, a couple of network cards, you
might get up to 90W or so avg. This would be on the low side
of the 1A range, which is consistant with your Alpha
figures.

A Versalogic VSBC-6 with a 300W K6 and 32MB EDO memory will
burn about 17W, figure maybe 20W or so with more memory. A
Toshiba 2GB, 2.5" EIDE drive will burn about 0.7A@5VDC,
for another 3.5W.  With a careful choice of fan, you could
probably get the entire system under 30W (typical). One
should probably allocate 50W per node to be safe. This
sort of works out to about 20A to 30A @ 220VAC for a
rack filled with 128 nodes.

Thus, on second look, the power consumption is pretty
much a wash between the two architectures. I really
was suprised at how little something like the DK440LX
board would chew up.

So, I guess that the main argument to be made here would
be that it would be a bunch easier to prepare the
rackmount power supplies and the feed harnesses for
an SBC-based system rather than an ATX-based system.

--Bob

-- 
============================================================
Bob Drzyzgula                             It's not a problem
bob@drzyzgula.org                until something bad happens
============================================================