parallel programming paradigm

Tony Skjellum tony at MPI-Softtech.Com
Thu Sep 21 18:28:48 PDT 2000

See below.

Anthony Skjellum, PhD, President (tony at 
MPI Software Technology, Inc., Ste. 33, 101 S. Lafayette, Starkville, MS 39759
+1-(662)320-4300 x15; FAX: +1-(662)320-4301;
"Best-of-breed Software for Beowulf and Easy-to-Own Commercial Clusters."

On Fri, 22 Sep 2000, Horatio B. Bogbindero wrote:

> i would just like to ask some question that have been bugging my mind:
> -is the master-slave model for parallel programming the most frequently
> used model?
It is hard to say, but these classes clearly exist
pure data parallel (like SCALAPACK, LINPACK, many parallel algorithms)
master slave model where slaves do DATA PARALLEL computations
master slave model where slaves do independent task parallel (or even
   work in smaller groups on distinct tasks; each subgroup could be
   data parallel for instance)
pure task parallel - lots of different stuff going on with lots of 
   heterogeneous communication

> -is SCALAPACK implemented as master-slave?
No, it is data parallel
> -is PETSc implemented as master-slave?
Most of the linear algebra in PETSc would be data parallel
> -does benchmark tests like LINPACK and HPL test for node-node or pairwise
> bandwidth on all the nodes?
LINPACK (and related HPL) test two things:
  reductions of single real numbers
  broadcasts of submatrices (single rows/columns and block row/columns)
  They do not explicitly do node-to-node or pairwise, nor overlapping
  of communication and computation

> -was is the most commonly used non master-slave model being use by you?
Data parallel is what most parallel libraries and many parallel programs
use: matrix-vector, matrix-matrix, LU, QR, data decomposition type
> thanks for your time. i am exploring the possibilities for different types
> of parallel programming paradigms.
> ---------------------
> william.s.yu at
> "... all the modern inconveniences ..."
> 		-- Mark Twain
> _______________________________________________
> Beowulf mailing list
> Beowulf at

More information about the Beowulf mailing list