[Beowulf] Lowered latency with multi-rail IB?

[Joshua mora acosta]

So a way to quantify if multirail helps on network latency driven workloads
there should be a sinthetic benchmark that can be built to show off the impact
of balancing these requests among multiple HCAs bound to different network
paths or core pairs like an all-to-[all,gather,scatter] or barrier benchmark
and in theory observe half the total latency of that overall communication.
So I think multirail will reduce latency of sinchronizations (collective
calls) but not to latency driven point to point communications.

Joshua

------ Original Message ------
Received: 12:37 PM CDT, 03/27/2009
From: Håkon Bugge <hbugge at platform.com>
To: Craig Tierney <Craig.Tierney at noaa.gov>Cc: Joshua mora acosta
<joshua_mora at usa.net>, DPHURST at uncg.edu,        beowulf at beowulf.org
Subject: Re: [Beowulf] Lowered latency with multi-rail IB?

> On Mar 27, 2009, at 18:20 , Craig Tierney wrote:
> 
> > What about using multi-rail to increase message rate?  That isn't
> > the same as latency, but if you put messages on both wires you
> > should get more.
> 
> Exactly why we saw almost 2x speedup on message rate (latency)  
> sensitive apps using Platform MPI. We call the technique alternating;  
> any communicating peer will use a HCA or port, but a single MPI  
> process will alternate between different HCAs (or ports), depending on  
> which peer he communicates with.
> 
> 
> Håkon