[Beowulf] typical latencies for gigabit ethernet

[Mark Hahn]

> Hmm...well I must be doing something terribly wrong then. Our
> latencies are in the 140 microseconds range (as revealed by ping)

well, ping is not _really_ a benchmark ;)
but it does sound like you have interrupt coalescing enabled.
on our dl145g2 nodes (BCM95721), I can peel ~40 us off ping times
by "ethtool -C eth1 rx-usecs 1".  ping is also reporting round-trip,
(MPI benchmarks quote half-rtt).  also, ping is using 64B packets,
not min-sized ones.

> What's a typical debug protocol? What should I be checking for?

well, look at "ethtool -c" output and try with "ping -s 20".