[Beowulf] copying data between clusters

[kyron]

On Fri, 05 Mar 2010 11:22:14 -0500, Mike Davis <jmdavis1 at vcu.edu> wrote:
> Michael Di Domenico wrote:
>> How does one copy large (20TB) amounts of data from one cluster to
>> another?
>>
>> Assuming that each node in the cluster can only do about 30MB/sec
>> between clusters and i want to preserve the uid/gid/timestamps, etc
>>   
> If the clusters are co-lo I wouldn't copy I would use shared storage. If

> they are not co-located I would use patience.
> 
> Seriously though, for a one time copy, I would consider copying to an 
> external system and then physically moving that system. To do this and 
> preserve ownerships you will need to duplicate accounts and groups.


...and we are all assuming non-compressibility; otherwise, use pbzip2 ;)