[Beowulf] Q: IB message rate & large core counts (per node)?

[Brice Goglin]

Greg Lindahl wrote:
>> now that I'm inventorying ignorance, I don't really understand why RDMA 
>> always seems to be presented as a big hardware issue.  wouldn't it be 
>> pretty easy to define an eth or IP-level protocol to do remote puts,
>> gets, even test-and-set or reduce primitives, where the interrupt handler
>> could twiddle registered blobs of user memory on the target side?
>>     
>
> That approach is called Active Messages, and can be bolted on to
> pretty much every messaging implementation. Doesn't OpenMX provide
> that kind of interface?
>   

Open-MX offers what MX offers: no explicit RDMA interface, only 2-sided.
But something similar to a remote get is used internally for large
messages. It wouldn't be hard to mplement some RDMA-like features in
such a software-only model like Mark said above.

Brice