[Beowulf] Performance tuning for Jumbo Frames

[Alex Chekholko]

On Sun, 13 Dec 2009 21:26:52 -0600
Rahul Nabar <rpnabar at gmail.com> wrote:

> On Sat, Dec 12, 2009 at 10:40 AM, Patrick Geoffray <patrick at myri.com> wrote:
> > Rahul,
> >
> > Rahul Nabar wrote:
> >>
> >> I have seen a considerable performance boost for my codes by using
> >> Jumbo Frames. But are there any systematic tools or strategies to
> >> select the optimum MTU size?
> >
> > There is no optimal MTU size. This is the maximum payload you can fit in one
> > packet, so there is no drawback to a bigger MTU.
> 
> Thanks! So I could push it beyond 9000 as well? Reason is I've seen a
> steady boost in performance so far. 1500 < 4000 < 9000.
> 
> Maybe my performance continues to increase beyond 9000 too?

Well, remember, your hardware has to support it, first.

I have a Foundry FGS648P switch which lists in the specs: "Jumbo Frames
up to 10,240 bytes for 10/100/1000 and 10GbE ports".  I turn that on by
issuing the command "jumbo frames" (then saving to flash, etc).

The 10GigE NICs that I have are the HP NC510C NetXen-based cards.  I
use the driver from the HP support site, and that driver only supports
up to 8000 MTU.  So I use 8000 bytes as my MTU. 

Set it as high as you can; there is no downside except ensuring all
your devices are set to handle that large unit size.  Typically, if the
device doesn't support jumbo frames, it just drops the jumbo frames
silently, which can result in odd intermittent problems.

-- 
Alex Chekholko   chekh at pcbi.upenn.edu