[Beowulf] How to know if infiniband network works?
hearnsj at googlemail.com
Thu Aug 3 08:59:50 PDT 2017
Faraz, do you mean the IPOIB tcp network, ie the ib0 interface?
Good question. I would advise joining the Openmpi list. They are very
friendly over there.
I have always seen polite and helpful replies even to dumb questions there
(such as the ones I ask).
I actually had to do something similar recently - we have nodes with only
IB, so I had to run OpenMPI over Infiniband,
but also say that the control connection had to use the ib0 interface.
On 3 August 2017 at 17:41, Faraz Hussain <info at feacluster.com> wrote:
> Thanks for everyone's help. Using the Ohio State tests, qperf and
> perfquery I am convinced the IB network is working. The only thing that
> still bothers me is I can not get mpirun to use the tcp network. I tried
> all combinations of --mca btl to no avail. It is not important, more just
> Quoting Michael Di Domenico <mdidomenico4 at gmail.com>:
> On Thu, Aug 3, 2017 at 10:10 AM, Faraz Hussain <info at feacluster.com>
>>> Thanks, I installed the MPI tests from Ohio State. I ran osu_bw and got
>>> results below. What is confusing is I get the same result if I use tcp or
>>> openib ( by doing --mca btl openib|tcp,self with my mpirun command ). I
>>> tried changing the environment variable: export OMPI_MCA_btl=tcp,self,sm
>>> Results are the same regardless of tcp or openib..
>>> And when I do ifconfig -a I still see zero traffic reported for the ib0
>>> ib1 network.
>> if openmpi uses RDMA for the traffic ib0/ib1 will not show traffic,
>> you have to use perfquery
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
>> To change your subscription (digest mode or unsubscribe) visit
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Beowulf