[Beowulf] Q: IB message rate & large core counts (per node)?

[Greg Lindahl]

On Tue, Feb 23, 2010 at 06:23:59PM -0500, Brian Dobbins wrote:

>   Actually, it's often *for* performance that we look towards hybrid
> methods, albeit in an indirect way - with RAM amounts per node increasing at
> the same or lesser rate than cores, and with each MPI task on *some* of our
> codes having a pretty hefty memory footprint, using fewer MPI processes and
> more threads per task lets us fully utilize nodes that would otherwise have
> cores sitting idle due to a lack of available memory.

If you have data structures which are identical in every process, you
might be able to mmap them shared, reducing your memory usage even for
the hybrid case.

I have only once before heard this kind of thing reported, and perhaps
it was you (can't remember who). I don't think it's a big driver for
hybrid programming.

Anyone else?

-- greg