[Beowulf] MPI_Isend/Irecv failure for IB and large message sizes

[Mark Hahn]

> I am running into problems when sending large messages (about
> 180000000 doubles) over IB. A fairly trivial example program is attached.

sorry if you've already thought of this, but might you have RLIMIT_MEMLOCK
set too low?  (ulimit -l)

> [[60322,1],1][btl_openib_component.c:2951:handle_wc] from b1 to: b2 error polling LP CQ with status LOCAL LENGTH ERROR status number 1 for wr_id 199132400 opcode 549755813  vendor error 105 qp_idx 3

105 looks like it might be an errno to me:
#define ENOBUFS         105     /* No buffer space available */

regards, mark.