[Beowulf] Re: Re: Home beowulf - NIC latencies

[Joachim Worringen]

Rob Ross wrote:
> Making a sequence of MPI_Isends followed by a MPI_Wait go faster than a 
> sequence of MPI_Sends isn't hard, particularly if the messages are to 
> different ranks.  I would guess that every implementation will provide 
> better performance in the case where the user tells the implementation 
> about all these concurrent operations and then MPI_Waits on the bunch.

In this case, the user should think about MPI_Alltoall(v) - there are 
MPI implementations which do this in a smarter way than 
Isend/Irecv/Waitall to achieve much better performance than using the 
naive approach. Especially if you go to large process numbers, some 
coordination can help a lot, even for a full bisection network like a 
single-stage full crossbar...

Generally, collectives are there to let the library know what kind of 
communication is coming next. All speculations in the library based on 
monitoring and predicting non-collective communication will probably 
only do good in the matching micro-benchmark (my personal experience).

  Joachim

-- 
Joachim Worringen - NEC C&C research lab St.Augustin
fon +49-2241-9252.20 - fax .99 - http://www.ccrl-nece.de