[Beowulf] Mutiple IB networks in one cluster

[Bogdan Costescu]

On Thu, Jan 30, 2014 at 5:33 PM, Prentice Bisbal
<prentice.bisbal at rutgers.edu> wrote:
> IB has more than enough bandwidth for
> message-passing and I/O.

Bandwidth is not the only measure for performance. Latency and number
of messages per unit of time that can be handled are others.
Especially the latter was the subject of a heated debate a few years
ago on this list.

I would suggest first finding out whether there is a real need of such
a setup and especially quantifying the need. F.e. do the applications
perform simultaneously message-passing and I/O ? This could be the
case with non-blocking MPI calls overlapping with classical or even
MPI I/O.

I could also see some effects for the (many) cases where the existing
IB uses a fat-tree topology and the parallel job uses some overloaded
IB links. But, as this was a theoretical question and mine a
theoretical answer, I'll stop here :)

Cheers,
Bogdan