[Beowulf] Re: dedupe Filesystem

[Kilian CAVALOTTI]

On Wednesday 03 June 2009 14:24:19 Lawrence Stewart wrote:
> I probably don't need to remind anyone here that deduplication on a
> live filesystem (as opposed to backups) can have really bad
> performance effects.  Imagine if you have to move the disk arms around
> for every file for every block of every file.  Modern filesystems do
> well at keeping files contiguous and often keep all the files of a
> directory nearly.  This locality gets trashed by deduplicaiton.  This
> won't matter if the problem is making backups smaller or making
> incrementals run faster, but it is not good for the performance of a
> live filesystem.

It's kind of new to me, but it looks like some vendors have block-level 
deduplication systems integrated in some of their product lines.
See http://www.theregister.co.uk/2009/06/05/dell_block_dedupe/ or 
http://www.theregister.co.uk/2009/06/01/quantum_extends_dxi/

> -Larry/thinking about what to do next

:(

Cheers,
-- 
Kilian