[Beowulf] Question on COAMPS, WRF and NHM

[Anand Vaidya]

We are in the process of acquiring a new cluster for running weather modelling 
software viz. NRL COAMPS, WRF and NHM (Japan)

We are currently running COAMPS on a Cluster of 50+ GigE and dual socket DC 
Opterons, NFS, CentOS4, RAM size=1GB/core, the performance seems to be 
limited by I/O (network I/O primarily). The performance flattens out at about 
32CPU.

Looking at the budget, current hardware availability, we have narrowed down to 
dual socket Intel Quad Cores, with 2GB/core and DDR infiniband, and CentOS 
5.x, OpenMPI 1.2.x, SGE 6.x (Or maybe we will buy faster D-DC AMDs)

We did enquire with the organizations regarding suitability of these, they 
could only offer limited help (understandably, the orgs may not be running 
the configs we intend to buy)

I do understand that factors such as grid size etc play a role. I am right now 
looking at gross factors before getting into actual test runs with different 
configs.

I would like to to whether any users of the aforementioned software can help 
answer the following questions:

- Does memory bandwidth (STREAMS?) have a significant impact? (Intel shared 
bus -vs- AMD's dedicated interconnect), since QCs worsen the shared bus 
loading

- Is infiniband worth it? (NRL seems to think it does enhance performance), 
however no additional details are available.

- Is a parallel filesystem (eg: Lustre, GPFS, GFS) vs NFS

Regards
Anand