[Beowulf] GPFS on Linux (x86)
atchley at myri.com
Thu Sep 14 09:43:55 PDT 2006
On Sep 14, 2006, at 12:14 PM, Gil Bloch wrote:
>> Hi Brent,
>> You do not mention what interconnect (protocol or advertised
>> throughput) that you are considering. If you are considering
>> gigabit Ethernet, you can skip the rest of this message.
>> If, however, you are considering something faster than gigabit
>> Ethernet, bear in mind that GPFS only can use TCP (as far as I
>> know). You will be limited to the performance of the TCP on your
>> interconnect. If you are using TCP/IP/Ethernet, then expect a very
>> high CPU load. If you are considering an IB solution, look closely
>> at their IPoIB results.
> Luster File System (open source from CFS) support native
> InfiniBand, delivering higher performance with low CPU utilization.
> Gil Bloch
> Mellanox Technologies
I had Lustre in mind when I wrote that, but I could not speak for the
other filesystems mentioned because I do not know about them. Also, I
did not want it to appear that I was recommending one over another.
<waving the corporate flag>
That being said I just submitted the code for Lustre over MX and the
results are quite good. See https://mail.clusterfs.com/wikis/lustre/
MX-10G. By the way, those performance numbers are using a Myrinet
switch, but I can get the same results using a low-latency Ethernet
switch like a Fujitsu XG700.
Starting the MX support for PVFS2...
</waving the corporate flag>
To be clear, my employer is agnostic about which filesystem people
use (i.e. Myricom does not prefer/recommend any particular solution)
as long as people buy our NICs (and switches when needed). :-)
More information about the Beowulf