[Beowulf] hpl size problems

[Mark Hahn]

> > yes, there are interrupts and the like, but you have to remember that 
> > modern machines have massive memory bandwidth, big, associative caches,
> > and such stuff doesn't matter much.
> 
> I was thinking that the increased context switching that would occur
> with more processes running would also increase the frequency at which
> the processes would bounce between the CPU's (no CPU/memory affinity).
> Now add to that the time it takes to repopulate the 2MB of L2 cache.

well, gratuitous flipping between cpus is certainly not good,
but Linux has had affinity (at least reluctance to flip) for a long time.
but even so, 2MB is not extreme: at 6 GB/s, it's only going to hurt if 
you do it a lot (such as every 20ms scheduling quantum.)  if you flipped
even 10 Hz, I'm not sure you'd notice...

I'd love to see numbers, if anyone has measured this!