[Beowulf] Lowered latency with multi-rail IB?

[Craig Tierney]

Joshua mora acosta wrote:
> The only way I got under 1usec in PingPong test or with
> ib_[write/send/read]_lat is with QDR and back to back (ie. no switch).
> With switch I get 1.1[3-7]usec [HP-MPI, OpenMPI, MVAPICH].
> It does not matter the MPI although I have to agree with Greg that multirail
> also increases latency.
> Multirail is used for:
> i) reliability
> ii) higher bandwidth
> 
> Best regards,
> Joshua
> 

What about using multi-rail to increase message rate?  That isn't
the same as latency, but if you put messages on both wires you
should get more.

A vendor tried to convince me recently that this was important,
but they had no benchmarks to back it up.

Craig


> 
> ------ Original Message ------
> Received: 11:11 PM CDT, 03/26/2009
> From: Greg Lindahl <lindahl at pbm.com>
> To: beowulf at beowulf.org
> Subject: Re: [Beowulf] Lowered latency with multi-rail IB?
> 
>> On Thu, Mar 26, 2009 at 11:32:23PM -0400, Dow Hurst DPHURST wrote:
>>
>>> We've got a couple of weeks max to finalize spec'ing a new cluster.  Has
> 
>>> anyone knowledge of lowering latency for NAMD by implementing a 
>>> multi-rail IB solution using MVAPICH or Intel's MPI?
>> Multi-rail is likely to increase latency.
>>
>> BTW, Intel MPI usually has higher latency than other MPI
>> implementations.
>>
>> If you look around for benchmarks you'll find that QLogic InfiniPath
>> does quite well on NAMD and friends, compared to that other brand of
>> InfiniBand adaptor. For example, at
>>
>> http://www.ks.uiuc.edu/Research/namd/performance.html
>>
>> the lowest line == best performance is InfiniPath. Those results
>> aren't the most recent, but I'd bet that the current generation of
>> adaptors has the same situation.
>>
>> -- Greg
>> (yeah, I used to work for QLogic.)
>>
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org
>> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
> 
> 
> 
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
> 


-- 
Craig Tierney (craig.tierney at noaa.gov)