[Beowulf] programming multicore clusters

[Joseph Mack NA3T]

I've googled the internet and searched the Beowulf archives
for "hybrid" || "multicore" and the only definitive 
statement I've found is by Greg Lindahl, 17 Dec 2004

"Most of the folks interested in hybrid models a few years 
ago have now given it up".

I assume this was from the era of 2-way SMP nodes.

Multicore CPUs are being projected for 15yrs into the 
future (statement by Pat Gelsinger, Intel's CTO, quoted in
http://cook.rfe.org/grid.pdf)

I expect the programming model will be a little different
for single image machines like the Altix, than for beowulfs
where each node has its own kernel (and which I assume will
be running dual quadcore mobos).

Still if a flat, one network model is used, all processes 
communicate through the off-board networking. Someone with a 
quadcore machine, running MPI on a flat network, told me 
that their application scales poorly to 4 processors. 
Instead if processes on cores within a package were working 
on adjacent parts of the compute volume and communicated 
through the on-board networking, then for a quadcore 
machine, the off-board networking bandwidth requirement 
would drop by a factor of 4 and scaling would improve.

In a quadcore machine, if 4 OMP/threads processes are 
started on each quadcore package, could they be rescheduled 
at the end of their timeslice, on different cores arriving 
at a cold cache? On a large single image machine, could a 
thread be scheduled on another node and have to communicate 
over the off-board network? In a single image machine (with 
a single address space) how does the OS know to malloc 
memory from the on-board memory, rather than some arbitary 
location (on another board)?

I expect everyone here knows all this. How is everyone going 
to program the quadcore machines?

Thanks Joe
-- 
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!