[Beowulf] copying data between clusters
scott at cse.ucdavis.edu
Fri Mar 5 10:21:01 PST 2010
> I'd like to paralyze that across multiple nodes to drive the aggregate up
> I was hoping someone would pop up say, hey use this magical piece of
> software. (of which im unable to locate)..
Sounds like what we are doing with hadoop and gridftp-hdfs for our LHC
cluster. Basically you would run N gridftp servers where N is the number
of nodes (and preferably uncontended uplinks as well) on the destination
cluster. Then run something like bestman (it'll act as a director).
It is a non-trivial stack of software but it should get the job done.
More information about the Beowulf