[Beowulf] Mature open source hierarchical storage management

[Nifty Tom Mitchell]

On Fri, Oct 23, 2009 at 04:12:11PM +1100, Carl Thomas wrote:
> 
>    HI all,
>    We are currently in the midst of planning a major refresh of our existing
>    HPC cluster.
>    It is expected that our storage will consist of a combination of fast
>    fibre channel and SATA based disk and we would like to implement a system
>    whereby user files are automatically migrated to and from slow storage
>    depending on frequency of usage. Initial investigations seem to indicate
>    that larger commercial�hierarchical storage management systems vastly exceed our budget.
> 
>    Is there any mature open source alternatives out there? How are other organisations dealing with transparently presenting different tiers of storage to non technical scientists?
> 

Just curious -- how large and how big are the deltas in the hierarchy?

I ask because the new generation of 2TB SATA disks appear
to be establishing the groundwork for a list of new storage
options including cluster file systems that run circles around
NFS and large storage RAIDS.