Network adapter.

[Jon Tegner]

Thanks a lot for comments and suggestions!

> I think that you have a problem with your switch :
> The maximum aggregate bandwidth available is too low to sustain high
> troughput transfer between more than 2 nodes. Which switch brand and
> model do you use for your cluster ?

I have done some further testing, using netpipe
(http://www.scl.ameslab.gov/Projects/ClusterCookbook/nprun.html). That
application sends packages of increasing size between two nodes, and
there is a drastic reduction in throughput when the size of the block
approaches around 5800 bytes. For 5780 bytes speed is 67.45 Mbps and
for 5800 bytes it is 1.34 Mbps. For increasing sizes it stays low (at
least up till 1e6 bytes (where I stopped)).

This is just for communication between two nodes, but since the
communication passes through the switch (D-link, des-3225G) it can
still be a result of a poorly configured one (no expert in that area),
but it can(?) still be a problem with the card.

I'll change to different cards on two of the nodes to check it out.

/jon