<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Looks interesting, thank you.<div class=""><br class=""><div class="">
Jeff Friedman<br class="">Sales Engineer<br class="">o: 425.420.1291<br class="">c: 206.819.2824<br class=""><a href="http://www.siliconmechanics.com" class="">www.siliconmechanics.com</a><br class=""><br class="">
</div>
<br class=""><div><div class="">On Feb 23, 2017, at 1:14 PM, Chris Dagdigian <<a href="mailto:dag@sonsorol.org" class="">dag@sonsorol.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">On Amazon you should be looking at CfnCluster:</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="https://aws.amazon.com/hpc/cfncluster/" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">https://aws.amazon.com/hpc/cfncluster/</a><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">The entire HPC stack is:</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">- Directly supported by AWS</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">- Written, used and deployed as a CloudFormation template</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">- Very easy to extend and customize</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">- Is under more active development than MIT StarCluster which is what we used to use for similar purposes</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Normally we use CfnCluster for AWS-only HPC but it should be amenable to a cloudbursting scenario especially if your VPN/VPC links are solid</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Chris</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">Jeff Friedman <<a href="mailto:jeff.friedman@siliconmechanics.com" class="">mailto:jeff.friedman@siliconmechanics.com</a>><br class="">February 23, 2017 at 3:56 PM<br class="">Thank you all for the info, it is very useful. It seems most of the cloud orchestrator software includes a bit more functionality than we need. We want to use the standard HPC provisioning, scheduling, and monitoring software, and just automate the setup and presentation of the cloud nodes. We are looking into establishing a VPN to AWS, and then continuing to see what software would do the best job of the automated setup/teardown of cloud resources. We are looking at just using AWS CloudFormation as an option. There is also Bright Computing Cluster Manager, Cycle Computing, RightScale, and a couple others. But again, I think these are a bit to robust for what we need. I’ll keep y’all posted if interested.<br class=""><br class="">Thanks again!<br class=""><br class="">Jeff Friedman<br class="">Sales Engineer<br class="">o: 425.420.1291<br class="">c: 206.819.2824<br class=""><a href="http://www.siliconmechanics.com" class="">www.siliconmechanics.com</a><br class=""><br class=""><br class="">On Feb 23, 2017, at 10:49 AM, Lev Lafayette<lev.lafayette@unimelb.edu.au> wrote:<br class=""><br class="">On Wed, 2017-02-22 at 10:02 +1100, Christopher Samuel wrote:<br class=""><blockquote type="cite" class="">On 21/02/17 12:40, Lachlan Musicman wrote:<br class=""><br class=""><blockquote type="cite" class="">I know that it's been done successfully here by the University of<br class="">Melbourne's Research Platforms team - but they are bursting into the non<br class="">commercial Aust govt Open Stack installation Nectar.<br class=""></blockquote></blockquote><br class="">In context that was after (a) small test cases of cloud bursting worked<br class="">and (b) cloud bursting was used to replace our existing cloud partition.<br class=""><br class=""><blockquote type="cite" class="">So now they just provision extra VM's when they need more and add them<br class="">to Slurm and given demand doesn't seem to go down there hasn't been a<br class="">need to take any away yet. :-)<br class=""></blockquote><br class="">Watch this space ;)<br class=""><br class=""><blockquote type="cite" class="">So this doesn't really reflect what Jeff was asking about as it's all<br class="">the same infrastructure, it's not hitting remote clouds where you have<br class="">to figure out how you are going to see your filesystem there, or how to<br class="">stage data.<br class=""><br class=""></blockquote><br class="">Very much so. The ability to set up an additional partition to external<br class="">providers (e.g., amazon, azure, any openstack provider) is much less of a<br class="">problem that the interconnect issues which are quite significant.<br class=""><br class=""><br class="">All the best,<br class=""><br class=""><br class="">Lev Lafayette <<a href="mailto:lev.lafayette@unimelb.edu.au" class="">mailto:lev.lafayette@unimelb.edu.au</a>><br class="">February 23, 2017 at 1:49 PM<br class="">On Wed, 2017-02-22 at 10:02 +1100, Christopher Samuel wrote:<br class=""><blockquote type="cite" class="">On 21/02/17 12:40, Lachlan Musicman wrote:<br class=""><br class=""><blockquote type="cite" class="">I know that it's been done successfully here by the University of<br class="">Melbourne's Research Platforms team - but they are bursting into the non<br class="">commercial Aust govt Open Stack installation Nectar.<br class=""></blockquote></blockquote><br class="">In context that was after (a) small test cases of cloud bursting worked<br class="">and (b) cloud bursting was used to replace our existing cloud partition.<br class=""><br class=""><blockquote type="cite" class="">So now they just provision extra VM's when they need more and add them<br class="">to Slurm and given demand doesn't seem to go down there hasn't been a<br class="">need to take any away yet. :-)<br class=""></blockquote><br class="">Watch this space ;)<br class=""><br class=""><blockquote type="cite" class="">So this doesn't really reflect what Jeff was asking about as it's all<br class="">the same infrastructure, it's not hitting remote clouds where you have<br class="">to figure out how you are going to see your filesystem there, or how to<br class="">stage data.<br class=""><br class=""></blockquote><br class="">Very much so. The ability to set up an additional partition to external<br class="">providers (e.g., amazon, azure, any openstack provider) is much less of a<br class="">problem that the interconnect issues which are quite significant.<br class=""><br class=""><br class="">All the best,<br class=""><br class=""><br class="">Christopher Samuel <<a href="mailto:samuel@unimelb.edu.au" class="">mailto:samuel@unimelb.edu.au</a>><br class="">February 21, 2017 at 6:02 PM<br class=""><br class="">We (I help them with this) did try cloudbursting into the Melbourne Uni<br class="">Openstack instance (the same place that provided the VM's for most of<br class="">Spartan) but had to give up on it because of a bug in Slurm (since<br class="">fixed) and the unreliability of bringing up VM's - from memory we had<br class="">one particular case where it tried to boot about 50 nodes and about 20<br class="">of them failed to start.<br class=""><br class="">So now they just provision extra VM's when they need more and add them<br class="">to Slurm and given demand doesn't seem to go down there hasn't been a<br class="">need to take any away yet. :-)<br class=""><br class="">So this doesn't really reflect what Jeff was asking about as it's all<br class="">the same infrastructure, it's not hitting remote clouds where you have<br class="">to figure out how you are going to see your filesystem there, or how to<br class="">stage data.<br class=""><br class="">cheers!<br class="">Chris<br class="">Lachlan Musicman <<a href="mailto:datakid@gmail.com" class="">mailto:datakid@gmail.com</a>><br class="">February 20, 2017 at 8:40 PM<br class="">I know that it's been done successfully here by the University of Melbourne's Research Platforms team - but they are bursting into the non commercial Aust govt Open Stack installation Nectar.<span class="Apple-converted-space"> </span><a href="http://nectar.org.au/" class="">http://nectar.org.au</a><br class=""><br class="">cheers<br class="">L.<br class=""><br class=""><br class=""><br class="">------<br class="">The most dangerous phrase in the language is, "We've always done it this way."<br class=""><br class="">- Grace Hopper<br class=""><br class=""><br class="">_______________________________________________<br class="">Beowulf mailing list,<span class="Apple-converted-space"> </span><a href="mailto:Beowulf@beowulf.org" class="">Beowulf@beowulf.org</a><span class="Apple-converted-space"> </span>sponsored by Penguin Computing<br class="">To change your subscription (digest mode or unsubscribe) visit<span class="Apple-converted-space"> </span><a href="http://www.beowulf.org/mailman/listinfo/beowulf" class="">http://www.beowulf.org/mailman/listinfo/beowulf</a><br class="">Jeff Friedman <<a href="mailto:jeff.friedman@siliconmechanics.com" class="">mailto:jeff.friedman@siliconmechanics.com</a>><br class="">February 20, 2017 at 7:03 PM<br class="">Hi all,<br class=""><br class="">Has anyone dealt with bursting from an on-prem HPC cluster to the cloud before? Are there any providers that stand out in this category? Quick searches reveal the usual suspects (AWS, Google, etc). Just wondered what real world experience has to say… :)<br class=""><br class="">Thanks!<br class=""><br class="">Jeff Friedman<br class="">Sales Engineer<br class="">c: 206.819.2824<br class=""><a href="http://www.siliconmechanics.com/" class="">www.siliconmechanics.com</a><br class=""><br class=""><br class="">_______________________________________________<br class="">Beowulf mailing list,<span class="Apple-converted-space"> </span><a href="mailto:Beowulf@beowulf.org" class="">Beowulf@beowulf.org</a><span class="Apple-converted-space"> </span>sponsored by Penguin Computing<br class="">To change your subscription (digest mode or unsubscribe) visit<span class="Apple-converted-space"> </span><a href="http://www.beowulf.org/mailman/listinfo/beowulf" class="">http://www.beowulf.org/mailman/listinfo/beowulf</a></blockquote></div></div><br class=""></div></body></html>