[Beowulf] Re: typical latencies for gigabit ethernet

[Gerry Creager]

Dave Love wrote:
> Gerry Creager <gerry.creager at tamu.edu> writes:
> 
>> Can you say something about any tuning you did to get decent results? 
> 
> To get the lowest latency, turn off rx interrupt coalescence, either
> with ethtool or module parameters, depending on the driver.  Of course,
> you may not want to turn it off completely, depending on how much load
> the extra interrupts cause.

I'll look at that and give it a try.  Thanks.

>> We've had some interesting problems (until lately; new broadcom drivers 
>> helped) getting decent performance.  We're using Dell 1950's with 
>> Harpertown and broadcom NICs are native.
>>
>> Which driver parms had to get tweaked?
> 
> There are (at least) two different drivers for Broadcoms, and I've no
> idea what that hardware has.  I've only used tg3, and am actually
> somewhat confused by it.  (I haven't had time to try to understand the
> driver code and I'm not primarily interested in those NICs as the
> nVidias on our boxes seem better.)  Are you using tg3 or bnx2?

I had rather nasty results with tg3 and abandoned it.  We're using bnx2 
now.  The latest iteration seems (guardedly) better than the last one.

gerry
-- 
Gerry Creager -- gerry.creager at tamu.edu
Texas Mesonet -- AATLT, Texas A&M University	
Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843