[Beowulf] Station wagon full of tapes

[Bogdan Costescu]

On Tue, 26 May 2009, Gerry Creager wrote:

> How can I get sufficient cloud resources for computing... ... that 
> will handle reasonable weather models with their small message MPI 
> chatter, and lots of file I/O?

By asking Amazon to equip their cloud nodes with a HPC interconnect or 
to have fat nodes (=many cores, use only shared memory for MPI). How 
would you do it differently for you own cluster ? Do you really think 
that Amazon has the magic powder to pour over Gigabit Ethernet and 
turn it into 1microsecond latency interconnect ?

> However, getting data into S3 for availability, when a daily 
> multi-gigabyte dataset is used for initiation, and another is 
> created as output, is going to be expensive, and likely slow.

I haven't tried to access S3; is the bandwidth limitation per account, 
per connection, per access node or some other way ? Striping comes to 
mind as a way to use several accounts/connections/access nodes to move 
data.

> Another potential use: A company or research group doesn't know if 
> they want to dip into HPC.

Argh, if they do this they might never dip into HPC ;-)

--
Bogdan Costescu

IWR, University of Heidelberg, INF 368, D-69120 Heidelberg, Germany
Phone: +49 6221 54 8240, Fax: +49 6221 54 8850
E-mail: bogdan.costescu at iwr.uni-heidelberg.de