[Beowulf] Station wagon full of tapes
gerry.creager at tamu.edu
Tue May 26 05:27:09 PDT 2009
There was an interesting brainstorming session at Rocks-A-Palooza a
couple of weeks ago. Someone wants to offer Amazon resources. Problem
remains for me: How can I get sufficient cloud resources for computing
(I'll hammer on dataset transport in a moment) that will handle
reasonable weather models with their small message MPI chatter, and lots
of file I/O? I've been assured that Amazon's ready to accommodate that.
However, getting data into S3 for availability, when a daily
multi-gigabyte dataset is used for initiation, and another is created as
output, is going to be expensive, and likely slow. I think there are
other approaches that have to be evaluated. I am not sure the cloud is
ready for MPI play on a significant basis, just yet.
I can envision it for small, embarrassingly parallel, low-communications
jobs. *atHome comes to mind, but they're already tapping spare cycles
-- at peoples home PCs -- and aren't likely to pay for this.
Another potential use: A company or research group doesn't know if they
want to dip into HPC. They can "rent" a virtual cluster in the cloud,
and if the results warrant, consider getting their own real cluster, or
buying into a campus infrastructure condominium model.
John Hearns wrote:
> There was a discussion recently on cloud computing, and how effective
> it would be for HPC.
> I made the comment that the time for getting large data sets to/from
> the cloud provider would be
> a limiting factor. I think Amazon was listening to me:
> They will accept hard drives shipped in the mail.
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
Gerry Creager -- gerry.creager at tamu.edu
Texas Mesonet -- AATLT, Texas A&M University
Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843
More information about the Beowulf