Just slightly off topic?

[Robert G. Brown]

On Wed, 16 Sep 1998, Goodin, Lee (AZ77) wrote:

> I have been reading the messages from this listserv system for some time.
> 
> I have some questions:
> 
> 1 - with all of the obsolete computers available,  like 486DX4-100' or early
> Pentium based computers, I'm wondering if
>      Beowulf technology would turn them into 'super computers'?

Yes, pending what you mean by "supercomputers".  I mean, you have to
accept a priori that a current 400 MHz PII is at least 10 times faster
(if not considerably more) than a 100 MHz 486.  So you have to buy 10
obsolete computers AND handle all the hassle of network IPC's and other
bottlenecks that have been considerably advanced in the meantime (e.g.
cache size, raw memory speed, memory bus bandwidth, etc. etc.) just to
break even -- on perfect code.  Price comparison suggests that 300 MHz
CPU's that are really probably 10x or more better (10x is absurdly
conservative for a 496-PII or Celeron comparison) can be had for ~1K.
Your price point for the 486's, fully networked, had therefore be
"free".  Even $100 each is too much to spend.  At free you won't really
win in performance, you'll just have fun.

With all that said, sure, you can hook a bunch of 486 boxes together on
a network and do useful work and even get 100's of MFLOPs, which was
"supercomputer" performance five or ten years ago.  Just remember that
the Cray 1 was 12 stunning MFLOPs (and my laptop would eat it for
breakfast today:-).  So, great idea for a high school learning project
or a bunch of kids who want to have fun -- bad idea from an economic or
performance point of view.

> 2 - How much of an increase in MIPS can a Beowulf system create?

Depends.  On too many things to list here.  On some tasks, a beowulf can
yield (very nearly) N*(mips or mflops/host) where N is the number of
participating hosts.  On others, N^(1/2)(speed/host), or
N^(1/7)(speed/host) or even worse (for example, NEGATIVE) scaling can
occur for all N, and usually will occur for large enough N regardless of
the problem.  Beowulfs are very sensitve to rate limiting bottlenecks in
distributed system design, which is why obsolete systems are "bad" --
you will PROBABLY get worse scaling for your stack of 486's than you
would for a rack of more modern boxes, because they have improved the
balance of various system bottlenecks as technology has advanced in the
meantime.

> 3 - What applications can be run on a Beowulf/Linux system?

Mostly ones you write yourself to accomplish some task you wish to
accomplish.  There are some "gee whiz" applications that are available
-- one of my favorites is the pvm/x mandelbrot set generator, where even
a modest cluster allows one to get "to the bottom" (of available pixel
resolutions) of the Mandelbrot set in two or three minutes.  This used
to take one of those Crays and days a decade or two ago...and they're
pretty!

> 4 - This is an I. T. NETWORK intensive technology, what certifications, and
> courses of study would enable the novice like myself
>       to make their own Beowulf systems?

Join the list, listen in, get the extreme linux CD.  Beyond that, you
will need to become a generic Unix/linux expert, which requires books
like Bach's "The Design of the Unix Operating System", Nemeth, Seabass
Snyder, Hein's "Unix System Administration Handbook", and various
O'Reilly texts on linux, networking, beowulfs (I think Don Becker just
finished a Beowulf O'Reilly book).  The book Linux Device Drivers by
Rubini I have found very useful.  Of course I assume that you know C,
C++, F77, and are literate in /bin/sh (or /bin/bash) and perl, sort of
minimally.

There are also lots of useful URL's posted on the list and on the
beowulf website that point to white papers, beowulf websites of various
exotic flavors, and somewhere there are list archives, which are worth
perusing.  If your beowulf is going to be made up of SMP systems you
should join the linux-smp list, otherwise the linux-kernel list and as
many other linux driver lists relevant to your hardware as you can
stand.  Sorry, information and help is plentiful but disorganized and
grazing it is time consuming.

> 5 - I've been considering changing careers,  what certifications  (if any)
> would allow me to go places in the I.T. industry?

Well, to my own experience "certification" is less important than
experience and drive.  If you build a decent beowulf and make it do
useful work, you will have completed a graduate degree in beowulfism
(high end networking, parallel theory and design, high end systems
engineering, etc.).  Obviously courses can help you learn parts of this,
although you can also learn them on your own.  Most of the working
systems administrators I know, paradoxically enough, were NOT computer
science students in college and many (including myself) have no
certification whatsoever (in CS -- my Ph.D. is in theoretical physics).
After being a systems administrator, systems programmer, systems
engineer, and beowulf owner/operator for fifteen years or so it sort of
ceases to matter.

Your biggest problem is finding somebody to pay you while you do an
apprenticeship.  Certification aside, all the folks I know who do this
who are NOT CS persons went through an apprenticeship phase with one or
more "gurus" mentoring them.  So, if you can enough experience at home
to get a job as a systems person, maybe at a University or small
company, then after two or three years you're on your way.

   rgb

Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb@phy.duke.edu