[Beowulf] Home beowulf - NIC latencies
ctierney at HPTI.com
Fri Feb 4 14:13:19 PST 2005
On Fri, 2005-02-04 at 13:20, Andrew Piskorski wrote:
> On Thu, Feb 03, 2005 at 04:53:27AM +0100, Vincent Diepeveen wrote:
> > Please note MPI is probably what i'll use, though i keep finding
> > online information about 'gamma'. Is that faster latency than MPI
> > implementations?
> Gamma is a non-TCP/IP Linux 2.6.x network driver for Intel Pro/1000
> gigabit ethernet cards, for use with MPI. It offers much better
> latency (11 us or so) than TCP/IP over ethernet (maybe 60 or 100 us),
> but worse than the specialized HPC interconnects (maybe 3 us).
See Josip's post on tweaking interrupts on gigE drivers, but
I have a small system with Intel gigE cards and a Dell gigE switch.
Latency between two nodes through the swtich is 30 us. This is
typical of what I see for other gigE cards. A latency of 60-100
is a bit high.
Avoiding TCP/IP is still a big improvement.
> The attraction of GAMMA, is that Intel Pro/1000 cards can be had for
> $11 to $60 or so each (depending on exact model, etc.), and gigabit
> switches are also pretty cheap, while SCI or Myrinet is somewhere in
> the $500 to $1500 per node range (I don't keep track).
> So if your application can benefit from lower latency, but you want
> something really cheap, GAMMA should be well worth trying.
More information about the Beowulf