Beowulf in a Box

[Mark Brinicombe]

On Mon, 28 Sep 1998, Kragen wrote:

> (I've forwarded this, and the post it's a reply to, to the sa-beowulf
> list because Doug Eadline is a commercial Beowulf builder.  His
> experience may be very useful in letting us know what to expect.)
> 
> On Mon, 28 Sep 1998, Douglas Eadline wrote:
> > So what happened? Well (In my opinion) INMOS could not keep
> > up with the other CPUs of the time 
> 
> This could conceivably happen with the StrongARM.  Intel is developing
> it, so I'm sure they *can* keep up if they want to; they've said they
> want to, but we all know how fast such things can change.

I believe Intel have already announce plans and some of their road map for
the StrongARM. Certainly I expect it to be around for a number of years.

> If I understand correctly, the ARM family is actually specced by Acorn,
> and several companies actually produce ARMs -- is that right?  Acorn
> seems to be refocusing on consumer electronics, so this might possibly
> be something to worry about.

The ARM process family is designed by ARM Ltd (Advanced RISC Machines) who
licenced the ARM core to Digital who enhanced to for the StrongARM and
transferred the licence to Intel. In addition they are several dozen chip
manufacturers who have licenced ARM cores for embedded devices e.g.
Samsung, Cirrus Logic, TI, VLSI etc.

(Just checking the WWW site, there are 30 silicon partners, Alcatel, AKM,
Atmel, Cirrus Logic, Epson, Hyundai, IBM, Intel, LG Semiconductor, LSI
Logic, Lucent Technologies, Matsushita, NEC, OKI, Philips, Mitel
Semiconductors, National Semiconductor, Qualcomm, Rockwell, Rohm, Samsung,
Sharp, Sony, Symbios Logic, Texas Instruments, VLSI Technology and
Yamaha).

The ARM processor started at Acorn years ago but was spun off as a
separate company in 1990. Since then ARM has grown considerably and Acorn
has shunk (ARM is at least 3 times as big).

As far as the use of ARM processors goes these days Acorn is a minority
user and there recent restructure was not completly unexpected and is
irrevant to the ARM processor and its market.

> Intel is announcing the next generation of ARM stuff next month; people
> say it will be 500MHz and have hardware floating-point.

Higher clocks speeds are expected. I believe ARM will be announcing their
next generation core at the next microprocessor forum.

> > Failed promises were the first problem of the Transputer,
> > but I think it's ultimate demise (other than an embedded CPU)
> > came from the lack of acceptance of "niche hardware" by the 
> > mainstream.  Sure the embedded system guys, love this kind of
> > stuff, but I believe it is tough sell to get someone
> > to use niche hardware for following reasons:
> > 
> > 1) single source hardware (possible overnight obsolescence)
> > 2) support comes from a single vendor and is limited (manpower
> >    pool is very limited)
> > 3) because of the single source nature an organization must
> >    make a large investment (of time and money) - this is the biggest
> >    problem.   
> 
> Well, I think the StrongARM is more like the Pentium than like the
> Transputer with regard to these characteristics.

ARM themselves do not manufacturer the processors but only licence the
design to semiconductor manufacturers. This allows then to concentrate on
what they are good at and leave the silicon to manufacturers with
fabrication plants. This I believe has helped to establish the ARM
processor world wide in conjunction with its low power consumption.

> 1) The vendor is Intel, but there are ARM chips from other vendors.
> The companies that have licensed the ARM core are Intel, NEC, Philips,
> National Semiconductor, Rockwell, Samsung, TI, Intel, IBM, Lucent, and
> 20 others, according to <URL:http://www.arm.com/CoInfo/CoBackground/>.
> As far as I can tell, nearly all of these companies actually make ARM
> processors, although most of them are embedded into other products, and
> many of them are rather slow -- for example, Cirrus Logic makes
> something called CL-PS7110, which is 15MIPS and 66mW.  More to the
> point, though, any high-speed, low-power-consumption CPU that can talk
> to 100MHz SDRAM and a PCI bus could easily be substituted, as soon as
> NetBSD or Linux was ported to it.
> 
> I think that, even independent of the market for this little
> supercomputer-on-a-PCI-card, high-speed ARM CPUs will be available for
> quite a while -- they're just used in too many things.

I would agree.

> 2) Well, see above.  Support for the ARM is pretty broad, and both
> Intel and ARM Inc. are developing high-speed ARMs.
> 
> 3) What's a large investment?  You'd probably have to port your
> software to NetBSD, which would probably not be a large investment if
> it currently runs on more than one Unix (other than Linux and Solaris,
> which are so similar that most apps run with only trivial changes).
> Testing your ported software could probably take a few weeks.  The
> hardware will probably cost less than getting another PC.
> 
> This is quite likely to be a smaller investment than buying another
> high-end machine.
> 
> (Of course, if you have to rewrite all your code to work in
> fixed-point, you *would* have a large investment.)
> 
> > There is a large amount of comfort knowing that you are not
> > relying on a single person, company, or product to run your machine.  
> 
> Agreed.
> 
> > I believe that one of the
> > reasons Beowulf/Clusters are very popular is that they
> > are "plug and play" replacements (from a software
> > standpoint) for much more expensive machines and therefore, the cost to
> > adopt clusters is small.  If you can deliver "plug and play"
> > to a market segment, then $/MIPS is a good sell.   
> 
> I think this is a plug-and-play proposition; it doesn't require
> rewriting applications in funny languages, modifying them to support
> new messaging styles, etc.  (Unless they're on a shared-memory
> multiprocessor machine.)  The worst case is that you can't do
> floating-point at any kind of reasonable speed, which locks out a big
> market segment.
> 
> > To say Intel is behind ARM helps a bit, but not much. Intel
> > killed their own children(i860/960) and closed their 
> > supercomputer shop (except for custom machines).  
> 
> True.

One rumor I heard was that it was Intel aquiring the StrongARM that
finally nailed the lid on the i960 coffin.

> > Like it or not, the "PC" is known concept, people are more
> > comfortable with things they know, than with better things
> > they do not know.
> 
> Certainly true.  Linux is running into the same problem.
> 
> > Finally, my guess as to the amount of work/cost involved to 
> > bring this idea market is rather large.
> 
> Well, it's not so much a matter of bringing it to market.  Simon Thorpe
> needed some better hardware for his neural-network stuff, and so he got
> in touch with the folks at Causality/Chaltech.  So they're designing
> and building the boards for him.  They wanted to sell a few more boards
> than Simon needed, in order to bring the cost down to something
> reasonable.  They've found enough customers for that.
> 
> It may be that some of the folks involved want to invest some work in
> getting this to a bigger market.
> 
> The amount of time should be a few months; if everything goes well (no
> redesigns, etc), the hardware will be available in November.

Things currently appear to be on-target.

> > It sounds as though
> > there is still some hardware/software  to be developed.
> 
> Yes.  Fortunately, the bulk of the software has been developed -- MPI,
> PVM, NetBSD, Linux, etc. -- and nearly all of the hardware (by the good
> folks at Digital); what remains is to put it together, write drivers,
> and fabricate the PC boards.

Certainly we have NetBSD and Linux is just about ready running on other
Chalice StrongARM hardware etc. and some software work is already underway
and hopefully will be coordinated with the hardware so the two can come
together around the same time. Certainly this is something where software
will continue to be developed and improved for some time.

> > Performance is uncertain, lots of assumptions.
> 
> True enough.  I don't know how fast the PCI bus will turn out to be,
> but I don't see how it could be terribly slow.  Even with 100
> processors, it would be as fast as switched 10Mbps Ethernet.  With
> smaller clusters (I say "clusters" because they're conceptually
> separate machines; they just happen to share a power supply and talk to
> each other over PCI) I think that performance will have to excel.

Some estimates can be made but certainly the communication performance is
not so easy to estimate and will probably vary depending on the
application.

> Simon says he's tested his neural-network code on StrongARMs already,
> so the CPU performance is not in doubt (at least for his application).
> I'm sure that other people could do the same.

The performance of Simons code on the StrongARM was very encouraging.

> > Is there a business plan?
> 
> I don't know.  Perhaps the companies actually producing these things
> have one.

Certainly we have a plan ;-)

> > By the time all this gets worked out, a lot may change.
> 
> True.  It looks like the ARM 10 processor will be released, which
> should be twice as fast and provide some floating-point hardware to
> boot.  It should be quick to design a new daughtercard for it.

This is a prime reason for keeping the system modular. It will allow
processors to be upgraded to newer or faster versions as they become
available.

Cheers,
				Mark