[Beowulf] Using Autoparallel compilers or Multi-Threaded libraries with MPI

[Håkon Bugge]

At Sat, 1 Dec 2007 15:15:31,Greg Lindahl <lindahl at pbm.com> wrote:
> > The SPEC HPG (High Performance Group) is having discussions about using
> > a hybrid of MPI and thread-level parallelism on the SPEC MPI2007
> > benchmark suite.
>
>I'd find it useful to debunk the notion that hybrid programming
>actually gives a speedup. That's probably not what HPG has in mind,
>but it'd be useful to the community.
>
>-- greg

I have a slightly different view. Hybrid 
programming is used for performance reasons, but 
only in cases where parallelization (to the same 
level) is impossible/impractical using the pure 
MPI mode, or the parallelization yields low 
efficiency. So, if you're able to achieve your 
performance with MPI, you probably will. But 
there are cases where you cannot; a) the 
"decomposition parallel efficiency" is not good 
enough or b) the processes need a huge (shared) table.

As to a), in the past I worked with a synthetic 
aperture radar application where I ended up with 
the hybrid model. The problem could only be 
decomposed in one dimension, and each process had 
33% overhead. Obviously, the hybrid model was a good choice in this case.

As to b), it might be more economic to size the 
memory on each node the the size of a single 
table and share it through shared memory. It is 
of course possible to share it from several MPI 
processes as well, but implementors might find 
their reason for using a hybrid model here.

Relevance to the SPEC MPI2007? To my knowledge, 
the applications here do not have any of the 
constraints above, so I would be severely 
surprised if anyone uses the hybrid model on them.


Håkon