[Beowulf] Torrents for HPC

[Bernd Schubert]

On 06/13/2012 03:55 PM, Joe Landman wrote:
> On 06/13/2012 09:40 AM, Bernd Schubert wrote:
>> On 06/09/2012 02:06 AM, Bill Broadley wrote:
>>>
>>> I've built Myrinet, SDR, DDR, and QDR clusters ( no FDR yet), but I
>>> still have users whose use cases and budgets still only justify GigE.
>>>
>>> I've setup a 160TB hadoop cluster is working well, but haven't found
>>> justification for the complexity/cost related to lustre.  I have high
>>> hopes for Ceph, but it seems not quite ready yet.  I'd happy to hear
>>> otherwise.
>>>
>>
>> What about an easy to setup cluster file system such as FhGFS? As one of
>> its developers I'm a bit biased of course, but then I'm also familiar
>> with Lustre, an I think FhGFS is far more easiy to setup. We also do not
>> have the problem to run clients and servers on the same node and so of
>> our customers make heavy use of that and use their compute nodes as
>> storage servers. That should a provide the same or better throughput as
>> your torrent system.

Arg, so many mistakes, why do I never notice those before sending the 
mail? :(

>
> I'd like to chime in and note that we have customers re-implementing
> storage with FhGFS.
>
> Ceph will be good.  You can build a reasonable system today with xfs as
> the backing store.  The RADOS device is an excellent basis for building
> reliable systems.

While the op does not need IB, most cluster nowadays do have IB. I think 
Ceph still does not support that, does it?


Cheers,
Bernd