[Beowulf] Re: Cluster Networking
d.love at liverpool.ac.uk
Sun Jun 28 04:45:09 PDT 2009
Rahul Nabar <rpnabar at gmail.com> writes:
> On Fri, Jun 26, 2009 at 1:30 PM, Jeff Layton<laytonjb at att.net> wrote:
>> Try something like OpenMX over GigE. Much better latencies
∼6μs, if that counts as much better.
>> and should perform and scale better.
Are there data on that? I'm not clear how much more efficient than TCP
it might be CPU-wise, for instance, and I'm not sure how best to check.
> How close does it get to native Myrinet performance? Or Infiniband.
Not at all for Infiniband. With the right NICs on two rails, it's
competitive with our Myrinet-2000 system. See open-mx.org for 10G data,
but they're presumably not relevant to you.
> OpenMX might be a great way for our cluster too to achieve better
> performance without changing our eth backbone.
In principle with Open MPI, it should use the two rails (NICs) to double
the bandwidth as with TCP; that's currently broken, although Manchester
seem to be getting away with it somehow. Brice will get back to fixing
it when he returns in a couple of weeks.
More information about the Beowulf