[Beowulf] dedupe filesystem

[Michael Di Domenico]

On Wed, Jun 3, 2009 at 6:10 AM, John Hearns <hearnsj at googlemail.com> wrote:
> I do agree with you that there are situations where you transfer the
> data to faster storage before running on it -
> I am thinking on one particular case right now!
> I Also agree with you that you then have the danger of 'squirreling
> away' copies on the fast storage, and forgetting why they are there.
> The systems administrator must put in place strong policies on this -
> leave your data on the fast storage, it gets deleted after N weeks.

Do you find such a policy hard to enforce with researchers?  I don't
have tiered storage today, but in the future i can see a need to have
a storage pool with SATA and a storage pool with SAS or faster drives
in it.  Some of the researchers where I am, work on data for months.
Is this something better solved with pre/post-amble copies or through
policies?