howto increase MTU size on 100Mbps FE

[Donald Becker]

On Fri, 26 Apr 2002, Joseph Mack wrote:

> I know that jumbo frames increase throughput rate on GigE and was
> wondering if a similar thing is possible with regular FE.

I used to track which FE NICs support oversized frames.  Jumbo frames
turned out to be so problematic that I've stopped maintaining the table.

> the MTU of 1500 was chosen for 10Mbps ethernet and was kept
> for 100Mbps and 1Gbps ethernet for backwards compatibility

Yup, 1500 bytes was chosen for interactive response on original
Ethernet.  (Note: originally Ethernet was 3Mbps, but commercial
equipment started at 10Mbps.)

The backwards compatibility issue is severe.  The only way to
automatically support jumbo frames is using the paged autonegotiation
information, and there is no standard established for this.

Jumbo frame *will* break equipment that isn't expecting oversized
packets.  If you detect a receive jabber (which is what a jumbo frame
looks like), you are allowed (and _should_) disable your receiver for a
period of time.  The rationale is that a network with an on-going
problem is likely to be generating flawed packets that shouldn't be
interpreted as valid.

> VLAN sends a packet larger than the standard MTU, having an
> extra 4 bytes of out of band data. The VLAN people have
> problems with larger MTUs. Here's their mailing list
> http://www.WANfear.com/pipermail/vlan/

Most of the vLAN people don't initially understand the capability of the
NICs, or why disabling Rx length checks is a Very Bad Idea.
There are many modern NIC types that have explicit VLAN support, and
VLAN should only be used with those NICs.  (Generic clients do not
require VLAN support.

> which indicate that the MTU is set in the NIC driver and
> that in some cases the MTU=1500 is coded into the hardware
> or is at least hard to change.

Hardware that isn't expecting to handle oversized frames might break in
unexpected ways when Rx frame size checking is disabled.  Breaking for
every packet is fine.  Occasionally corrupting packets as a counter
rolls over might never be pinned on the NIC.

The driver also comes into play.  Most drivers are designed to receive
packets into a single skbuff, assigned to a single descriptor.  With
jumbo frames the driver might need to be redesigned with multiple
descriptors per packet.  This adds complexity and might introduce new
race conditions.  Another aspect is that dynamic Tx FIFO threshold
code is likely to be broken when the threshold size exceeds 2KB.  This
is a lurking failure -- it will not reveal itself until the PCI is very
busy, then Boom...

> I don't know whether regular commodity switches (eg Netgear
> FS series) care about packet size, but I was going to
> try to send packets over a cross-over cable initially.

Most switches very much care about packet size.  Consider what happens
in store-and-forward mode.

All of these issues can be fixed or addressed on a case-by-case basis.
If you know the hardware you are using, and the symptoms of the
potential problems, it's fine to use jumbo frames.  But I would never
ship a turn-key product or preconfigured software that used jumbo frames
by default.  It should always require expertise and explicit action for
the end user to turn it on.

-- 
Donald Becker				becker at scyld.com
Scyld Computing Corporation		http://www.scyld.com
410 Severn Ave. Suite 210		Second Generation Beowulf Clusters
Annapolis MD 21403			410-990-9993