[Beowulf] Re: Re: Home beowulf - NIC latencies
patrick at myri.com
Tue Feb 15 01:48:09 PST 2005
Joachim Worringen wrote:
> Patrick Geoffray wrote:
>> While we are at it, here is my wish list for the next MPI specs:
>> a) only non-blocking calls. If there are no blocking calls, nobody
>> will use them.
> While this makes sense technically, nobody will probably offer an MPI
> implementation without MPI_Send for the next 20 years for compatibility
> reasons, so we can just forget about it.
Throw away compatibility. If you keep the legacy API, you have no
incentive for change. I don't want MPI-3, I want MPI-light. We are
against a wall because the MPI spec was too rich and developers took the
The weight of legacy will make shared memory paradigms the only proposal
for the next step. If you believe we have to support the whole MPI
semantics in the next message passing standards, then we are doomed.
>> c) ban of the ANY_SENDER wildcard: a world of optimization goes away
>> with this convenience.
> I think this could best be achieved with an assertion like those for
> one-sided and I/O. There are situations where ANY_SENDER is needed, or
> at least avoids large programming overheads.
It's used because it's there, there is no other reason. If you don't
know who sends you what in a message passing application, then you
cannot get either performance or robustness. If really you cannot do
otherwise (and I don't believe that), you can always use unexpected
messages (post the receive after Probe()ing), That's ugly, but you get
what you deserved :-)
>> d) throw away the user defined datatypes, or at least restrict it to
>> regular strides.
> This is nonsense: user-defined datatypes do not cause any overhead if
> you don't use them, there are ways to implemenent them very efficiently,
> and you can't do without in many situations (like MPI-IO).
I know this item would itch, you spend a lot of time working on that.
If you don't use user-defined datatypes, then you don't need it and it
should not be there in the first place. It's a temptation, it's too
easy. No, there is no ways to implement them efficiently unless they are
regular, and this is what I am willing to keep: strided types with long
segments. Everything else leads to memory copies. The developer should
wipe his own bottom instead of asking the message passing interface to
work around bad data layout. Sending a column of blocs, yes, that's
regular stride and it makes a lot of sense. Sending non-contiguous
irregular structure ? As we used to say in France, $100 and a chocolate
bar with that ?
Oh, BTW, I would gut MPI-IO and make a separate interface. Only a small
subset of applications use it and the core semantics are quite different
that pure message passing. Man, it's not MPI, it's emacs...
>> e) get rid of one-sided communications: if someone is serious about
>> it, it uses something like ARMCI or UPC or even low level vendor
> Instead, I propose to rework the MPI one-sided communications for a more
> simple and flexible semantic. The current definition does not match
> todays network capabilities, but was designed to allow a simple
> implemenentation for slow/non-RDMA networks.
I don't know about that. I just would took it out of the Message Passing
Interface because it's not message passing. There would certainly be a
need for a pure RMA interface, and there is already a lot of existing
work and experience to build upon.
>> Rob, you are politically connected, could you make it happen, please ?
> One person alone can't do this. The best place to discuss such things is
> the MPI users group meeting (EuroPVM/MPI, this year in Capri/Italy).
Nothing that radical would ever come out of EuroPVM/MPI (I heard that
Capri is a really nice place, I will definitively beg my boss) or any
other users group.
> Also, adding mpi.h to the standard to define an ABI is a good thing.
Just achieving that would be beyond my greatest expectations. It would
certainly be fun to watch. We could organize fist fights on the beach in
More information about the Beowulf