[Beowulf] mixing of Infiniband HCAs in cluster
kus at free.net
Fri May 26 09:46:32 PDT 2006
In message from "Gil Bloch" <gil at mellanox.co.il> (Wed, 24 May 2006
>> We have a pair of IB 4x (Mellanox MTLP23108) HCAs used currently for
>> direct switchless connection of pair of nodes.
>> Now we plan to buy 8-port (MTK37900) or 24-port IB switch w/HCAs,
>> which may be also DDR IB. And we want to build this cluster
>> also old pair of nodes w/MTLP23108 HCAs. But is this possible - to
>> have both HCA kinds (DDR and usual IB 4x HCAs) connected to the same
>> switch ? If the answer is "yes", is it necessary to setup all the
>> DDR-HCA drivers manually for work w/"4x speed" or all the necessary
>> things will be done via negotiations ?
>InfiniBand defines auto-negotiation of link speed, ...
>The only thing you should notice is that streams from DDR HCAs to SDR
>HCAs should reduce data rate to avoid flooding the switches. This is
>done on a per-stream (QP) base, thus the DDR port can be fully
>utilized. Using CMA for connection settings will take care of it
>your application does not use CMA, you should take care of it somehow
>(e.g. use CMA).
Sorry, what's CMA - Connection Manager ? In any case I don't know
about using of CMA if the application uses MVAPICH for parallelization
>> Mikhail Kuzminsky
>> Computer Assistance to Chemical Research Center,
>> Zelinsky Institute of Organic Chemistry
>> Beowulf mailing list, Beowulf at beowulf.org
>> To change your subscription (digest mode or unsubscribe) visit
>Beowulf mailing list, Beowulf at beowulf.org
>To change your subscription (digest mode or unsubscribe) visit
More information about the Beowulf