[Beowulf] Home beowulf - NIC latencies
Douglas Eadline, Cluster World Magazine
deadline at clusterworld.com
Mon Feb 7 07:34:44 PST 2005
On Fri, 4 Feb 2005, Andrew Piskorski wrote:
> On Thu, Feb 03, 2005 at 04:53:27AM +0100, Vincent Diepeveen wrote:
> > Please note MPI is probably what i'll use, though i keep finding
> > online information about 'gamma'. Is that faster latency than MPI
> > implementations?
> Gamma is a non-TCP/IP Linux 2.6.x network driver for Intel Pro/1000
> gigabit ethernet cards, for use with MPI. It offers much better
> latency (11 us or so) than TCP/IP over ethernet (maybe 60 or 100 us),
> but worse than the specialized HPC interconnects (maybe 3 us).
The "60-100 us" is incorrect. With proper tuning an e1000 can get 25us
latency (using netpipe). (see Jossip's post about tunning parameters)
Oh, and by the way this was using a 32 PCI desk top card.
A low latency number is not the whole story however, processor load
is another issue. The point is that tuning can make a difference. Default
values are usually set for maximum throughput and low CPU overhead.
It all depends on what your application needs. If you need GAMMA,
then that is a good choice, but many applications may work well
with proper tuning of NIC parameters.
As an aside, Netgear used to sell a low cost desktop NIC
(GA302T-tigon3/Broadcom) which had very good numbers as well.
I profiled this NIC in the first issue of ClusterWorld.
> The attraction of GAMMA, is that Intel Pro/1000 cards can be had for
> $11 to $60 or so each (depending on exact model, etc.), and gigabit
> switches are also pretty cheap, while SCI or Myrinet is somewhere in
> the $500 to $1500 per node range (I don't keep track).
> So if your application can benefit from lower latency, but you want
> something really cheap, GAMMA should be well worth trying.
Editor-in-chief ClusterWorld Magazine
Fax: 610.865.6618 www.clusterworld.com
More information about the Beowulf