[Beowulf] multi-threading vs. MPI

Geoff Jacobs gdjacobs at gmail.com
Sat Dec 8 09:35:03 PST 2007

richard.walsh at comcast.net wrote:
>     -------------- Original message --------------
>     From: "Toon Knapen" <toon.knapen at gmail.com>
>     How come there is almost unanimous agreement in the
>     beowulf-community while the rest is almost unanimous convinced of
>     the opposite ? Are we just tapping ourselves on the back or is MP
>     not sufficiently dissiminated or ... ?
>     Mmm ... I think the answer to this is that the rest of world
>     (non-HPC world) is in a time
>     warp.  HPC went through its SMP-threads phase in the early-mid 1990s
>     with OpenMP, and then we needed more a more scalable approach
>     (MPI).  Now that multi-core and multi-socket has brought parallelism
>     to the rest of the Universe, SMP-based parallelism has had a
>     resurgence ... this has also naturally caused some in HPC to revisit
>     the question as nodes have fattened. 
>     The allure of a programming model that is intuitive, expressive,
>     symbolically light-weight,
>     and provides a way to manage the latency variance across memory
>     partitions is irresistable.
>     I kind of like the CAF extension to Fortran and the concept of
>     co-arrays.  The co-array is
>     and array of identical normal arrays, but one per active
>     image/process.  They are defined as such:
>               real, dimension (N) [*] ::  X, Y
>     If the program is run on 8 cores/processors/images the * becomes 8. 
>     8, 1D arrays of size
>     N are created on each processor. In any references to the locale
>     component of the co-array
>     (the image on the processor referencing it), you can drop the []s
>     ... all other references (remote)
>     must include it.  This is symbolically light, but reminds the
>     programmer of every costly non-
>     local reference with the presence of the []s in the assignment or
>     operation.  There is much
>     more to it than that of course, but as the performance gap between
>     carefully constructed
>     MPI applications and CAF compiled code shrinks I can see the later
>     gaining some traction
>     for purely programming elegance related reasons.  If you accept that
>     notion that most MPI
>     programs are written at a B- level in terms of efficiency then the
>     idea of gap closing may not
>     be so far fetched.  CAF is supposed to be include in the Fortran
>     2008 standard.
>     rbw
>     -- 
>     "Making predictions is hard, especially about the future."
>     Niels Bohr
>     -- 
>     Richard Walsh
>     Thrashing River Consulting--
>     5605 Alameda St.
>     Shoreview, MN 55126 

But isn't CAF (and UPC, and Titanium) implicitly message passing for a
Beowulf anyway? It's attractive because it simplifies the process and
might be able to optimize communication, but underneath everything it's
still message passing.

Geoffrey D. Jacobs

To have no errors
  would be life without meaning
  No struggle, no joy

More information about the Beowulf mailing list