[Beowulf] Re: dedupe filesystem

[Joe Landman]

Gerry Creager wrote:

>> Ob-Beowulf:  You can run Venti on GNU/Linux,² but I don't know how the
>> current implementation performs.  Also, GlusterFS has a `data
>> de-duplication translator' on its roadmap, which I didn't see mentioned.
> 
> Our initial results with a GlusterFS implementation led us back to NFS. 
>  Who's got a really successful GlusterFS implementation working?
> 

We are seeing generally very good results with GlusterFS.  Performance 
is in part in how you arrange your translators, how your fabric is 
connected, etc.  If you don't work hard with it, you can get ~200 MB/s 
per client over SDR IB.  If you work a bit harder you can get ~500 MB/s 
per client.  You do need enough server bandwidth to be able to source 
the data.

I think I had sent a note in the past offering some assistance with 
this.  My bad if the note didn't get through ... or if my responses were 
lost in the shuffle.



-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics,
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/jackrabbit
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615