[Beowulf] high %system utiliziation on infiniband nodes

[Walid]

Hi,

I have two nodes Interconnected using Infiniband, and using Intel-MPI over
dapl1.2.7 from OFED 1.3.1 compiled localy on the same build, when there is
interconnect communication i can see on one of the nodes that i monitoring
have a high cpu utiliztion (%system) that exceeds 60%. the mpi job is
helloworld/pallas runing over two nodes, 8 cores each (16 processes in
total)

a snapshot of mpstat -P ALL on one node
06:22:20 PM  CPU   %user   %nice %system %iowait    %irq   %soft   %idle
intr/s
06:22:22 PM  all   30.25    0.00   69.75    0.00    0.00    0.00    0.00
1768.50
06:22:22 PM    0   30.00    0.00   70.00    0.00    0.00    0.00    0.00
566.50
06:22:22 PM    1   30.50    0.00   69.00    0.00    0.00    0.00    0.00
201.00
06:22:22 PM    2   30.50    0.00   69.50    0.00    0.00    0.00
0.00      0.00
06:22:22 PM    3   29.50    0.00   70.50    0.00    0.00    0.00
0.00      0.00
06:22:22 PM    4   28.50    0.00   71.00    0.00    0.00    0.00
0.00      0.00
06:22:22 PM    5   30.00    0.00   70.00    0.00    0.00    0.00
0.00      0.00
06:22:22 PM    6   31.00    0.00   69.50    0.00    0.00    0.00    0.00
1000.50
06:22:22 PM    7   32.00    0.00   68.00    0.00    0.00    0.00
0.00      0.00

now i get the same behaviour on RHEL5.0/5.1 and RHEL4.6, using Infiniband or
ethernet, so is this normal, to me it does not, or at least i have never
seen such behaviour before? the node is a DELL PE1950

regards

Walid
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20080726/998be018/attachment.html>