Beowulf - Single Board Computers? (long)

[Greg Lindahl]

> perhaps 128 CPUs per 40U rack, allowing
> room for network switches and power supplies.

OIC -- I hadn't realized you had something so aggressive in
mind. That's a pretty tough standard to meet, because your ability to
have any kind of disk or expansion networking cards or anything else
is very limited.

> > The 3rd company I know of in the
> > cluster market, Paralogic (www.xtreme-machines.com), seems to only
> > sell a relatively compact conventional case with cards at...
> 
> That might be interesting; their website doesn't give much in the
> way of inside-the-box pictures or diagrams, and they only list
> commodity motherboards. Do you have more information from them?

I don't have any more info; however, the picture at their website of
an "8 Pentium II cpu machine" looks like 4 conventional cases.

> Correct. I am only familiar with the Industrial SBC market. Do you
> have a list of URLs for Military vendors? I'd like to poke around
> in those. 

www.altatech.com -- Alpha/Myrinet CompactPCI
www.mc.com
www.sky.com -- PowerPC, custom switched network

> LAN-on-PCI may not be the cat's meow for current PCIs,
> but it seems to me that the future holds some promise,

Well, my crystal ball isn't any better than anyone else's, but
ethernet switches are lightyears ahead of PCI busses, and are likely
much higher volume. Yes, PCI interface silicon is high volume, but
99.99% of them are going to go into desktops which are happy with a
single bus. Still, we should keep in mind that the only thing that
makes Myrinet expensive is the interface cards, not the switches...

> This being said, I'm interested in where your 25%
> figure comes from...  do you expect this to be a
> limitation of the PCI architecture itself or simply a
> limitation of bus architectures in general?

It's just a WAG (wild a** guess) based on the difficulty of bursting
across PCI in the first place... if you aren't starting with a burst,
how can you turn it into a burst in order to efficiently switch it?
Ethernet packets are much more dull than PCI bus transactions. But,
I would be happy to be proven wrong.

> They may be
> cheap, but traditional motherboards take up a bunch of
> space for stuff that is little or no use in a Beowulf;
> having to run 110VAC to each processor is kind of a
> drag,

I'm not sure that AltaTech's commodity motherboard gizmo actually uses
that many power supplies. The thing which makes it big is the disk and
the ability to plug in a PCI card. Presumably if you had a motherboard
with no disk and didn't need any cards because you were using built-in
ethernet, you could get them to space the shelves closer than usual,
giving you about double their current density of 32 systems per rack.

And it would be cheap. And it's only twice as big as your idea system.

Being as naieve about power as I am, would this beast use considerably
less power if you had a consolidated gizmo for dishing out DC to each
motherboard? My existing alphas with their 300? or 400? watt power
supplies actually pull only 1 amp of 110V with just the cpu in use and
2 amps when they are using their disk drives. We measured this at the
breaker box using one of those induction gizmos. If getting rid of the
separate power supplies and not having a disk gets us down to 0.5 amps
per CPU, we're getting close to your ideal number with commodity
parts.

-- greg