[Beowulf] NWChem 5.1.1 + Infiniband OFED 1.3 + GA 4.1.1

[Joshua mora acosta]

Hello
I am trying to get to work NWChem 5.1.1 + Infiniband OFED 1.3 + GA 4.1.1 using
HPMPI (or any other MPI) and PGI or any other MPI.
I get the well known problem of not working over the network.
Here you have the configuration I am using just in case someone spots the
error.
It runs fine in node but not across the network.

Thanks,
Joshua

user1 at amdqc01:~/Chem_work/nwchem-5.1.1> more compile_nwchem.csh
#!/bin/csh
set echo

#module load hp-mpi
#module load pgi64/7.2

setenv LARGE_FILES TRUE
setenv LIB_DEFINES "-DDFLT_TOT_MEM=16777216"
setenv TCGRSH /usr/bin/ssh
setenv NWCHEM_TOP "$HOME/Chem_work/nwchem-5.1.1"
setenv NWCHEM_TARGET "LINUX64"
setenv TARGET "LINUX64"
setenv SCRATCH_DEF_DIR "\'/tmp\'"
setenv F77 pgf77
setenv CPP pgcpp
setenv CXX pgcpp
setenv FC pgf90
setenv CC pgcc
setenv NWCHEM_MODULES "all"
setenv USE_MPI y
setenv USE_MPIF y
setenv LIBMPI "-lmpi -L /usr/lib64 -libumad -lpthread"
setenv MPI_LOC /opt/hpmpi
setenv MPI_LIB /opt/hpmpi/lib/linux_amd64
setenv MPI_INCLUDE /opt/hpmpi/include
setenv HAS_BLAS yes
setenv BLAS_I8 yes
setenv BLAS_LIB "-L /opt/acml4.2.0/pgi64_int64/lib -lacml -lacml_mv"
setenv BLASOPT "-L /opt/acml4.2.0/pgi64_int64/lib -lacml -lacml_mv"
setenv ARMCI_NETWORK OPENIB
setenv MA_USE_ARMCI_MEM yes
setenv IB_HOME /usr/
setenv IB_INCLUDE /usr/include
setenv IB_LIB /usr/lib64
setenv LD_LIBRARY_PATH /opt/acml4.2.0/pgi64_int64/lib:$LD_LIBRARY_PATH
setenv NWCHEM_NWPW_LIBRARY /sn01/user1/home/NWChem/data/
setenv NWCHEM_BASIS_LIBRARY_PATH /sn01/user1/home/NWChem/data/libraries/
cd src
make FC=pgf90 nwchem_congig >& make_config.log
make FC=pgf90 all >& make.log
echo Done