<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On May 26, 2009, at 10:20 AM, "Robert G. Brown" <<a href="mailto:rgb@phy.duke.edu">rgb@phy.duke.edu</a>></div><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; ">Subject: Re: [Beowulf] Station wagon full of tapes<br>To: Jeff Layton <<a href="mailto:laytonjb@att.net">laytonjb@att.net</a>><br>Cc: Beowulf Mailing List <<a href="mailto:beowulf@beowulf.org">beowulf@beowulf.org</a>><br>Message-ID: <<a href="mailto:alpine.LFD.2.00.0905260946000.4021@localhost.localdomain">alpine.LFD.2.00.0905260946000.4021@localhost.localdomain</a>><br>Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed<br><br>On Tue, 26 May 2009, Jeff Layton wrote:<br><br><blockquote type="cite">I haven't seen the cloud ready yet for anything other than embarrassingly<br></blockquote><blockquote type="cite">parallel codes (i.e. since node, small IO requirements). Has anyone seen<br></blockquote><blockquote type="cite">differently? (as an example of what might work, CloudBurst seems to be<br></blockquote><blockquote type="cite">gaining some traction - doing sequencing in the cloud. The only problem<br></blockquote><blockquote type="cite">is that sequencing can generate a great deal of data pretty rapidly).<br></blockquote><br>I'm pretty skeptical of commercial rent-a-cluster business models.<br></span></blockquote><div><br></div><div>To me "Cloud" implies "embarrassingly parallel" infrastructure. High speed interconnects (which I would loosely define as "better than std GbE" are not cloud friendly... and really require a "Cluster for Hire" arrangement. Also, there's no "throat to choke" in the clouds, but my customers know how to find me to let me know when something isn't fair or could be done better for their specific use case.</div><br><div>It's actually a great business when you have the right mix of clients and expectations. IRS depreciation rules make it tough to survive... but this is a double edged sword since depreciation rules drive some of the bleeding edge users to external systems so they can stay bleeding edge every year without internal accounting fights.</div><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "></span></blockquote><br><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; ">Is this true in cluster computing? Nearly every cluster computer user I<br>know of wants "infinite capacity", not finite capacity. They are<br>limited by their budget, not their needs. Give them more capacity,<br>they'll scale up their computation and finish it faster or do it bigger<br>or both. </span></blockquote><div><br></div><div>This is where the "value" of a 2nd or 3rd year cluster really comes through for "value" researchers.... would they rather spend their money on a few systems for a long run, or a lot of systems for a short run? Since many researchers are "project" based and funded, access to a large system for short bursts of time could help a single researcher work through more "projects" or ideas faster than on his own simple cluster even if the "Per hour" charge is more... they are able to accomplish more with their time and work through more brilliant ideas... We can work out the "time value" of money, but what is the "time value" of a brilliant mind waiting for an answer? This is the reason Departmental or Enterprise class HPC systems *should* be the minimum scale an organization builds in house. Some commercial ISV's will often donate the temporary licenses to help test scaling limits for their customers before a big purchase is made.</div><br><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; ">Plus edge cases -- somebody that needs a cluster desperately, but only<br>for six months and to do one single computation (how common is that?<br>not very, I think).</span></blockquote><div><br></div><div>Six months is a long time. We routinely see requests for hundreds of compute cores in the < 1 Month range. They aren't "edge" or "desperate", they just understand the "inertia" of their internal systems group, vendors etc. and "peak shaving" is the best way to keep their in house system from being crushed under the weight of the queues.... while justifying longer term increases in capacity in house. Outsourced computing for this shouldn't be thought of as "all or nothing", it's just an added tool for the Researcher and his IT staff to manage when it fits their needs.</div><div><br></div><div>The researcher who needs the extra cycles may not be the one who is ultimately computing off site either.... if they have peers on the internal system that can more easily deal with the limits of network bandwidth then bartering can ensue and help everyone get more done without blowing the budget or wasting a lot of wall clock time. Ultimately Fedex is the highest bandwidth network, with terrible latency. We have rarely had to resort to this "fall back" network so far but are glad to know it is there and simple to manage.</div><div><br></div><div>In the end, it's all about Balancing the Bleeding edge user's needs with the Value edge user's needs. If we can load the teeter totter up with enough of each, much more research will be accomplished at a better overall value for all the researchers. The business of selling cycles on shared systems has been around since the first "virtual machines" hit the mainframe market long ago. It's been called many things, and funded many ways, but in the end scaling up to a specific plateau is more efficient assuming the machine stays reasonably loaded over it's useful life. </div><div><br></div><div>Researchers able to suffer occasionally long availability delays and "preemption" will find some really good deals backfilling our systems. This way we can have our utilization and our availability (for unexpected jobs) too. Some day it should be considered an ethical and ecological crime to have unused cycles wasted during the short life of a research computer. It should be an implicit goal for every researcher who controls a system to keep it doing useful work for themselves or their peers.</div><div><br></div><div>Full Disclosure: I am a part owner of a "cluster for hire" business trying to assemble the "wasted" cycles under so many desks to benefit all the researchers who won't miss the wirr and buzz under there desks... Heck, we'll even put our nodes under their desk if that's what they need us to do :)</div><div><br></div><div>Sincerely,</div><div>Greg W. Keller</div><div>R Systems NA, inc.</div><div><br></div></div><br><div><br></div></body></html>