[Beowulf] Multiple NIC on a node

[Patrick Geoffray]

Peter St. John wrote:
> I don't get it? I would have thought that if a large package were split
> between two NICs with two cables, then assuming the buffering and
> recombination at each end to be faster than the transmission, then the
> transmission would be faster than over a single cable? You don't mean that

The problem is ordering of packets and TCP. When you send a single TCP 
stream over two (or more) paths, then some packets will arrive 
out-of-order at the destination. TCP really does not like out-of-order 
packets and performance takes a (big) hit.

That's why most channel bonding mechanisms balance multiple streams over 
multiple NICs and send each stream on a single NIC. Other protocols than 
TCP may not have this problem if they don't require strict ordering for 
performance.

Patrick