channel bonding rtl8139 works??

[Martin Siegert]

On Wed, Sep 05, 2001 at 06:38:38AM -0400, Robert G. Brown wrote:
> On Mon, 3 Sep 2001, <<   Dragon  >> wrote:
> >
> >   Hi, does any body know if the RTL-8139 kernel module for the KNE120TX
> > works with channel bonding??
> >
> >  I channel bonded 3 KNE120tx cards on 2 computers (3 cards per system) and
> > the comunication between them is very slow, I don't know if the problem is
> > in the rtl8139 kernel module (Red Hat 6.2)
> 
> I don't know about channel bonding per se, but the RTL8139 NIC is Dark
> Evil.  In my opinion, of course: YMMV, caveat emptor, I may be crazy
> (some would say there is little doubt:-), standard disclaimers etc.

I used RealTek cards with the rtl8139 kernel module a while back (kernel
2.2.16, 2.2.17 when the root holes in those kernels were still unknown) for
channel bonding in a test configuration.
There was a problem with those cards as the setting of the MAC addresses
to the MAC address of eth0 did not work correctly. Hence, channel bonding
failed miserably (basically 2 out of 3 packets were dropped). A workaround
was published on this list
http://www.beowulf.org/pipermail/beowulf/2000-October/010236.html
You had to copy the MAC addresses manually from eth0 to eth1, eth2, etc.
After that channel bonding worked.
I do not know whether with newer versions of the driver this is still
necessary.

> Still, my own experience:  It is one of the few cards with which I can
> still consistently crash a linux box currently in my possesion when I
> whack it with a very heavy packet stream.  

I agree. When I said "channel bonding worked" that really meant it worked
for certain applications. When you run embarrassingly parallel jobs almost
any NIC will work, but then you wouldn't setup channel bonding in the first
place. For other applications that require higher bandwidth the rtl8139 did
succeed in hanging up the network once in a while. And if you run NFS (udp)
traffic over the channel bonded connection - you better don't do that:
Under high load the packet loss was so high that it could hang up the whole
box.

> I should note in saying this
> that this is with the current RH 7.1 kernel with its "stolen" 8139too
> driver, not Don's, although many kernel revisions ago I managed to crash
> boxes with two RTL8139's with Don's driver as well.
> 
> You should connect to the scyld website and visit the rtl8139 page and
> read the notes there.  I'd have to say that even if your particular
> implementation of the 8139 is more stable than my own (maybe Kingston
> did a better job of engineering the NICs than my no-name mfr, or maybe
> you're using Don's driver already and it actually works stably where the
> 8139too dies) it is a relatively poor choice for a beowulf NIC.  Even
> when I run just one card at a time (and cannot crash the system) the
> card seems to choke up under a heavy packet load and actually slow down
> to a crawl.
> 
> Crush.  Kill.  Destroy.  Choose another card.

Agreed. I don't let those cards come close to my cluster anymore.
If you want to do channel bonding, you do it for performance. 3 Realtek
cards under some circumstances only had a slightly better performance as
a single 3c905b in my tests - it just doesn't make sense.

> They are more expensive, but 3c905's do PXE and WOL and can save you the
> cost of a floppy or CD-ROM (per node) for the original install.  They
> have MUCH better latency and bw numbers in my tests -- better than even
> midrange cards that DON'T really suck (e.g. PNIC based cards).

Martin

========================================================================
Martin Siegert
Academic Computing Services                        phone: (604) 291-4691
Simon Fraser University                            fax:   (604) 291-4242
Burnaby, British Columbia                          email: siegert at sfu.ca
Canada  V5A 1S6
========================================================================