[Beowulf] Infiniband - Is there a HowTO?

[Joe Landman]

Hi Bill

Bill Wichser wrote:
> I have recently acquired some IB adapters and an IB switch.  My kernel 
> is 2.6.9-34.ELsmp and has a directory for infiniband under 
> kernel/drivers in /lib/modules.  No modules get loaded by default at 
> boot time though.
> 
> I can see the IB detected with lspci yet dmesg shows nothing.
> 
> 03:07.0 PCI bridge: Mellanox Technologies MT23108 PCI Bridge (rev a1)
> 04:00.0 InfiniBand: Mellanox Technologies MT23108 InfiniHost (rev a1)
> 
> 
> I'm just wondering how to proceed.  I imagine that I'll need to load 
> some modules and configure the interface with ifconfig.  Load up the 
> OpenIB RPM and build my OpenMPI libs.

Easiest way would be to use the Mellanox IBG2 
(http://www.mellanox.com/products/ibg2.php you have to register to 
download) set to build the RPMs/drivers for your kernel.  I did this 
recently for a different Mellanox card.  Once this is done, and you have 
configured your /etc/modprobe.conf, you might need to modprobe ib_mthca, 
then modprobe your ib0 interface, and run depmod -a.

> I just can't seem to find some HowTo to make sure I get it all correct. 
>  What modules to load, when.  That sort of thing.  If anyone could 
> provide me with some pointers at least I can get started on getting 
> something configured and tested.

The install script from the Mellanox tarball automates most of this.  A 
little hand editing as I indicated above, but it is mostly painless 
(good job Mellanox!)

Joe

> 
> Thanks,
> Bill
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit 
> http://www.beowulf.org/mailman/listinfo/beowulf

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web  : http://www.scalableinformatics.com
phone: +1 734 786 8423
fax  : +1 734 786 8452
cell : +1 734 612 4615