[Beowulf] Performance tuning for Jumbo Frames
patrick at myri.com
Tue Dec 15 20:41:59 PST 2009
Bogdan Costescu wrote:
> long as it fits in one page. At another time, the switch was more
> likely to drop large frames under high load (maybe something to do
> with internal memory management), so the 9000bytes frames worked most
> of the time while the 1500bytes ones worked all the time...
This is an important point. The way hardware flow-control works in
Ethernet, a switch has to be able to buffer two full frames plus the
time on the wire for the round-trip. For the curious, the PAUSEs packets
are sent in-band and you cannot send or receive partial frames.
So, instead of requiring ~4K per port minimum, you need about ~20K per
port. Add to that up to 8 priorities with DCB and the buffering
requirement are quickly getting out of hand. That's one big drawbacks of
large MTUs, along with contention with wormhole switching.
More information about the Beowulf