[Beowulf] Mellanox IB problem: xp0 module ?

Mikhail Kuzminsky kus at free.net
Mon Oct 18 09:10:42 PDT 2004

           Dear colleagues !

I've some problem w/Mellanox IBHPC-0.5.0 software inst (in particular, 
the absence of xp0 kernel module) on "standalone" node which isn't
connected currently w/IB switch or other node w/IB device.

I've installed Infiniband HCA (PCI-X Infinihost MT23108 low profile)
to upgarde my interconnect from GigEth to IB.

Tyan S2880/Opteron 242 under SuSE Linux 9.0 (2.4.21-243) is used as
the node for this installation. It's official software platform 
supported by whole Mellanox IBHPC-0.5.0 software collection
(it includes , in aprticular, THCA-3.2 driver). Software
environment is "fixed" because of a set of binary applications 
requiremnets, so last IBHPC-1.6.0 looks as inappropriate for as. 
1)After minor source modification (in mosal.c) the the IBHPC 
installation (INSTALL script) was finished successfully. IPoIB 
parameters setting was also performed in the frames of INSTALL script

2)But after finish of INSTALL and reboot I see that

a) mst tools started successfully
b) and I see then following boot messages:

Setting up network interfaces :
eth1  - both done
ib0: modprobe: modprobe: can't locate module xp0

and ib0 interface is down (I should note that IB cable isn't connected 
to HCA really). But I may do ib0 "up" manually; in particular, 
/etc/init.d/network start
put ib0 in "up" state.

I didn't find xb0.o  in /lib/modules/..., and in any Mellanox software
rpm's also ! I don't know what do xp0 module and where I may
found it :-(  Any reccomendations/ideas are welcome !

(FYI: some IB things like FLINT verification are OK, and opensm & mst
started successfully).

2) I configured IPoIB at IBHPC installation. (To try IBsNice) I issued
vapi start

after boot, and then I see in particular the message

Loading mod_ib_mgt FAILED

"Manual" modprobe mod_ib_mgt leads to the message
init_module: device or resource busy

If I run IBsNice.sh, then I receive the same message about

but IBsNice creates virtual eth2 , and ping to the IP of eth2 works

I'll be very appreciate if somebody clarify me this situation 
w/mod_ib_mgt. May be it's simple because of some misconfiguration of 
some IB software component ? 
(I didn't configure anythings after running of INSTALL script).

Mikhail Kuzminsky
Zelinsky Institute of Organic Chemistry

More information about the Beowulf mailing list