[Beowulf] Beowulf Application in the Sciences

[Robert G. Brown]

On Wed, 22 Sep 2004, Timo Mechler wrote:

> Hello all,
> 
> Through my studies of computer clusters so far, it has become apparent to
> me that a lot of applications exist in both the Math and Physics fields
> (e.g. different types modeling, for instance in high energy physics,
> etc.).  I'm now really wondering what kind of applications exist in the
> other physical sciences such as biology and chemistry.  How would clusters
> be benefical in these areas?  I would greatly appreciate if someone could
> provide me with a few examples and some insight.

Curiously enough, Cluster World Magazine devoted an issue quite recently
to computational chemistry.  Cluster computing is alive and well there,
especially in areas that overlap in various ways with physics, e.g.
quantum chemistry and molecular dynamics, as well as structural
modelling and more.

Biology, too, has many clusters serving many purposes.  There are
"biogrid" projects that range from departmental to national in scope;
see e.g. http://www.ncbiogrid.org/ for North Carolina's biogrid project
home page.  These projects typically focus on e.g. genomics and
bioinformatics, but there are also clusters that I'm aware of at Duke
that study e.g. conduction patterns in heart tissue (solving the poisson
equation in inhomogeneous media in a distributed way), biomolecular
folding, and a lot of other things in smaller projects more like what
one would typically find in math or physics or chemistry.

If you google a bit, see if you can scare up a copy of the relevant CWM,
and ask around at area universities or ask the contacts at things like
biogrid or chemstry grid projects, you can doubtless refine you
knowledge of this as much as you like.  Clusters and their larger
multidisciplinary variants the grids are firmly established in pretty
much all the hard sciences and engineering these days, to the point
where many campuses (like Duke) are experiencing explosive growth in the
number and type of clusters.  

Duke has its own internal grid project and a center that has recently
been set up to coordinate and house and permit sharing of clusters
purchased by individual groups across the sciences, engineering and
medicine (see www.csem.duke.edu).  This center can barely keep up with
the growth as groups all over campus are getting twenty nodes here, a
hundred nodes there.  It coordinates cluster management and facilitates
computational science work done on grids and ways of managing
distributed resources -- most of its toplevel staff have graduate
experience and/or degrees in a variety of sciences and have some decades
of cluser experience between them.  Many of them are frequent
contributors to and familiar names on this list.

This is actually a very exciting time for cluster computing -- it is
undergoing something of a paradigm shift from the "beowulf" model that
has somewhat dominated the last eight years back into a model that is
both older (distributed parallel computing and compute farms of various
sorts predated beowulf per se by many years) and newer (they never had
to cope with authentication, security, batch job coordination over a
wide range of projects with a wide range of resource constraints being
run on a heterogeneous collection of resources contributed AND OWNED by
many individuals with their own agendas and usage priorities;-).

It is amusing to contemplate where this is going.  In ten years will
there just be "the Grid" as there is now the Internet, where we install
a toplevel application to join our computer(s) to the grid and
transparently offer up idle cycles to others in exchange for
transparently picking up their idle cycles in return (all nice and
secure and sandboxed)?  Will we start to see a Gibsonesque turn, where
rogue/viral programs released into this grid become a strange form of
virtual "life"?

Stay tuned to this channel...

(Damn, I need to just put these replies straight into my CWM column.
Kill two birds with one stone, so to speak;-).

> 
> My other question is in regards to parallel programming.  Are there some
> free parallel programs out there that one can download to run on a
> cluster, or are most of the programs run on clusters "homegrown"?

There are plenty of free parallel programs you can run for fun or
specific purposes, some commercial parallel programs ditto, and yes,
there are lots of homegrown ones.  PVM and MPI sites will typically come
with some of the former or contain links to them, big projects (like
biogrid stuff) attract some commercial effort although there are also
frequently GPL tools available as well.  Homebrew abounds mostly because
a topically focussed physics computation (for example) is almost always
a one-off -- you code your particular bit of science in competition with
everybody else.  There are exceptions to this, of course, where groups
use the same basic tool to look at different things, but it's pretty
common to write or port your own code.

   rgb

> 
> Thanks in advance for your help.
> 
> -Timo Mechler
> 
> -- 
> Timo R. Mechler
> mechti01 at luther.edu
> 
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
> 

Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu