[Beowulf] InfiniBand channel bundling?
prentice.bisbal at rutgers.edu
Thu Oct 30 11:55:36 PDT 2014
You're talking only about latency, right? From the graphs FDR/Gen3 and
QDR/Gen3 are almost identical.
For bandwidth, QDR/Gen3 provided only a 16 - 20% improvement over
QDR/Gen2., depending on whether they were measuring one-way or
bidirectional bandwidth, whereas FDR seemed to provide about 40-50 % of
the bandwidth performance difference between FDR/Gen3 and QDR/Gen2.
Interestingly, the text of the article didn't compare FDR/Gen3 to
QDR/Gen3, but it's clear from the graphs that FDR/Gen 3 is still way
above QDR/Gen3 despite the 16-20% improvement Gen3 provides.
On 10/30/2014 11:08 AM, Brice Goglin wrote:
> Do you want to compare FDR vs QDR when used in the exact same PCIe
> slots? FDR often goes in Gen3 while QDR has been in Gen2 for a while.
> According to the link below, Gen3 is responsible for pretty much all the
> improvement between QDR/Gen2 and FDR/Gen3.
> Le 30/10/2014 15:41, Prentice Bisbal a écrit :
>> If end-to-end is lower for FDR, then what latency is being measured
>> for FDR that is higher than for QDR? According to Wikipedia, and the
>> Mellanox website, FDR does have a better latency than QDR (0.7
>> microseconds vs. 1.3 microseconds), but I and others on this list have
>> heard that FDR has worse latency than QDR. Have we been misinformed,
>> or does it depend on how you measure or define latency?
>> On 10/29/2014 06:46 PM, Gilad Shainer wrote:
>>> End-to-end FDR latency is lower than end-to-end QDR latency - per
>>> published measurments that can be found in multiple places.
>>> -----Original Message-----
>>> From: Beowulf [mailto:beowulf-bounces at beowulf.org] On Behalf Of Jörg
>>> Sent: Wednesday, October 29, 2014 3:45 PM
>>> To: Beowulf Mailinglist
>>> Subject: Re: [Beowulf] InfiniBand channel bundling?
>>> Hi all,
>>> thanks again for the wealth of information.
>>> Now, given that I am not interested in transporting files over the IB
>>> network but I am doing parallel calculations, I would have thought
>>> that the latency here is more important than the speed?
>>> Thus, if FDR has a higher latency than QDR, does that mean my
>>> performance is decreasing when I am running a calculation between nodes?
>>> For those of you who are into Chemistry code: I am using VASP, cp2k,
>>> quantum espresso and cpmd mainly. All of that is plain wave code.
>>> All the best from a wet London
>>> On Mittwoch 29 Oktober 2014 Prentice Bisbal wrote:
>>>> On 10/28/2014 04:43 PM, Mark Hahn wrote:
>>>>> On Tue, 28 Oct 2014, John Hearns wrote:
>>>>>> Here is a very good post from Glenn Lockwood regarding FDR versus
>>>>>> dual-rail QDR:
>>>>> indeed, very nice. though also quite surprising - is it known that
>>>>> FDR is so terrible for latency? seems astonishing to me.
>>>> Yes, it was known to me. I had already known that FDR was worse than
>>>> QDR for latency, but I don't remember my source. I don't know if I'd
>>>> characterize it as "so terrible", though.
>>> Dr. Jörg Saßmannshausen, MRSC
>>> University College London
>>> Department of Chemistry
>>> Gordon Street
>>> WC1H 0AJ
>>> email: j.sassmannshausen at ucl.ac.uk
>>> web: http://sassy.formativ.net
>>> Please avoid sending me Word or PowerPoint attachments.
>>> See http://www.gnu.org/philosophy/no-word-attachments.html
>>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
>>> To change your subscription (digest mode or unsubscribe) visit
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
More information about the Beowulf