low-latency high-bandwidth OS bypass user-level messaging for commodity(linux) clusters with commodity NICs(<$200), HELP! (GAMMA/EMP/M-VIA/etc.)

[Patrick Geoffray]

On Tue, 2002-12-17 at 10:13, Donald Becker wrote:

> Today it would be much more difficult to reproduce the exact
> environment.

Tell me about it :-) It really takes some time to realize the number of
parameters that involved in the big equation. Neglect one component and
part of the performance goes to the toilettes. This is a fragile
environment.
Still, a 50% difference is unusually large, specially with a code that
is OS-bypass, does not use interrupts and where the large part of the
critical path is running on a hardware independent of the host.

To come back on the original post, the fact that
Os-bypass/zero-copy/whatever communication layer is often tight to a
specific hardware is not only a way to secure an effective source of
revenue. It's also aimed at make the life of the software developers
much easier: having a common closed hardware environment reduces the
dependencies on the host. 
One big question when designing a non-TCP layer for Ethernet is how deep
to go trying to exploit the hardware ? If you try to be generic, you
will quickly realize that the existing driver architecture makes a very
decent job. If you starts to use hardware specific functionalities, you
will either lock yourself in a few hardware solutions (scary when you do
not control the future of the hardware line) or the amount of work
needed to support a large set of GigE NICs at a ssuch low level is
exploding. Add to that the fact that GigE chipsets have a quite short
life cycle and the vendors are reluctant to provide details about their
chips, I understand why there is no such product today.

Patrick
-- 

Patrick Geoffray, Phd
Myricom, Inc.
http://www.myri.com