Archives


- Beowulf
- Beowulf Announce
- Scyld-users
- Beowulf on Debian

[Beowulf] dedupe filesystem

Many of your questions may have already been answered in earlier discussions or in the FAQ. The search results page will indicate current discussions as well as past list serves, articles, and papers.

Search

Rahul Nabar rpnabar at gmail.com
Fri Jun 26 09:14:55 PDT 2009


On Tue, Jun 2, 2009 at 11:56 PM, Matt Lawrence<matt at technoronin.com> wrote:
> I have found a great deaal of duplication in install trees, particularly
> when you just want to install the latest.  I've managed to get some massive
> savings with NIM on AIX and some lesser but stll very good savings with
> CentOS by building parallel trees and hard linking the files.

I'm jumping on this discussion late but I found a great number (but
not so much in size I suspect) of dupes when I ran fdupes on my drives
now. Some in quite unexpected places. eg. we have job submission shell
wrappers for PBS/torque. The number of dupes indicates that often
people are not removing these files at all after the job finishes. So,
more than an automated-dedup solution fdupes has helped me know what
are the filesystem cleanup opportunities I have!

The next time someone comes running for a disk quota increase I know
what to check! ;-)

-- 
Rahul




More information about the Beowulf mailing list