[Beowulf] More multiple things per node

[Joachim Worringen]

Mark Hahn wrote:
> I think this is actually referring to binding a process to a cpu, no?
> messing with cpu-nic bindings doesn't really seem like MPI's business,
> since it knows the CPUs only through a socket...

You are right. However, the MPI library should make sure that the processes 
schedule their communication in the best possible way. If it makes a difference 
on a multi-NIC system (and the MPI library should know better than the user), it 
should make sure that i.e. each process always uses the same NIC for small 
messages. This depends much on the platform and thus should not be coded in the 
application, but in the platform-dependent MPI library.

> the sched_setaffinity interface seems simple enough, but if using it makes 
> a big difference, then IMO you've found a scheduler bug.  certainly the
> normal scheduler will not move processes around willy-nilly...

I've seen large differences in benchmarks when running MPI (MP-MPICH) on Windows 
NT4 on dual-SMP machines w/ or w/o process binding. This was some years ago, 
though. As always, the issue is the cache reuse.

  Joachim

-- 
Joachim Worringen - NEC C&C research lab St.Augustin
fon +49-2241-9252.20 - fax .99 - http://www.ccrl-nece.de