[Beowulf] GPFS on Linux (x86)

[Scott Atchley]

On Sep 14, 2006, at 12:14 PM, Gil Bloch wrote:

>> Hi Brent,
>>
>> You do not mention what interconnect (protocol or advertised  
>> throughput) that you are considering. If you are considering  
>> gigabit Ethernet, you can skip the rest of this message.
>>
>> If, however, you are considering something faster than gigabit  
>> Ethernet, bear in mind that GPFS only can use TCP (as far as I  
>> know). You will be limited to the performance of the TCP on your  
>> interconnect. If you are using TCP/IP/Ethernet, then expect a very  
>> high CPU load. If you are considering an IB solution, look closely  
>> at their IPoIB results.
> Luster File System (open source from CFS) support native  
> InfiniBand, delivering higher performance with low CPU utilization.
>
> Gil Bloch
> Mellanox Technologies

Hi Gil,

I had Lustre in mind when I wrote that, but I could not speak for the  
other filesystems mentioned because I do not know about them. Also, I  
did not want it to appear that I was recommending one over another.

<waving the corporate flag>
That being said I just submitted the code for Lustre over MX and the  
results are quite good. See https://mail.clusterfs.com/wikis/lustre/ 
MX-10G. By the way, those performance numbers are using a Myrinet  
switch, but I can get the same results using a low-latency Ethernet  
switch like a Fujitsu XG700.

Starting the MX support for PVFS2...
</waving the corporate flag>

To be clear, my employer is agnostic about which filesystem people  
use (i.e. Myricom does not prefer/recommend any particular solution)  
as long as people buy our NICs (and switches when needed). :-)

Scott