<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">I agree that the billing is non-trivial issue. In the last couple of years we’ve put quite a lot of effort into developing cost models, capacity planning as well planning for this kind of possible surplus. This enables us to track the costs at much finer granularity than just annual checkpoints and adjust things if necessary. <div><br></div><div>The customers are also starting out fairly small so it’s possible to evolve this as the service matures.</div><div><div><br></div><div>It’s also possible to fairly flexibly add capacity to react to demand, initially from excess unused capacity of our regular cluster and buying or leasing nodes in moderate batches. In the future I could imagine that HPC cluster nodes could also be “retired" to the cloud for a while before EOLing them. </div><div><br></div><div>The above things are quite crucial to develop and stabilize the chargeback model without having those hugely deep pockets. </div><div><br></div><div>I see that a major challenge is the friction between the funding models with the new way that services are produced: </div><div><br></div><div>For the end user the “remotely accessible shared resource" offers the potential for efficiently growing their services organically, setting up parallel test and dev environments on demand etc. For the more forward-looking contractors that are doing (pardon the buzzword) DevOps, these are already crucial features for developing services. However, for many public and research funding entities the focus is still towards infrastructure investments and fairly waterfally, rigid long-term capacity plans. </div><div><br></div><div>Going a bit offtopic here I guess. Sorry.. :)</div><div><br></div><div>O-P<br><div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">-- <br>Olli-Pekka Lehto<br>Development Manager, Computing Platforms </div><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">CSC - IT Center for Science Ltd.<br>E-Mail: <a href="mailto:olli-pekka.lehto@csc.fi">olli-pekka.lehto@csc.fi</a> // Tel: +358 50 381 8604 // skype: oplehto // twitter: @ople<br></div></div>
</div>
<br><div><div>On 14 May 2015, at 02:05, Lux, Jim (337C) <<a href="mailto:james.p.lux@jpl.nasa.gov">james.p.lux@jpl.nasa.gov</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div lang="EN-US" link="blue" vlink="purple" style="font-family: Helvetica; font-size: 14px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><div class="WordSection1" style="page: WordSection1;"><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">Without getting into the semantics of clouds, smoke, fog, or mirrors.<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">Isn’t this basically a “remotely accessible shared resource” which happens to be a cluster (defined as something more than a bunch of PCs on the same network: typically with a high performance interconnect and some “cluster management” software of one sort or another).<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">In all, a useful concept. The tricky part is how the “chargeback” system works. What we have here at JPL are called “service centers” for things like antenna ranges, equipment loan pools, clean room services, etc. They charge a “per unit” charge (where the unit depends on the service, be it day of use, month of rental, etc.) that per unit charge is determined at the beginning of the fiscal year by the manager of the service, based on past history, and is designed to cover all the operating costs of the service.<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">Of course, at the end of the year, if the total cost of operation is different than the total unit charges received, there’s a problem. And, strangely, I’ve never gotten a rebate from the service center because the TCO was less than they collected.<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">In any case, this kind of strategy is pretty common for “big iron” computers (when you used to lease a machine from IBM, you’d pay by the CPU-second, by the kilocore-second, etc.)…<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">It also will pass muster for government contracting, which requires that costs be allowable, accountable, and allocable (i.e. you can’t artificially reduce your profit by charging yourself exorbitant rates for computing services)<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">But it depends on having someone with deep enough pockets to absorb the instantaneous differences between revenue and expense (and the political expertise to handle the problem of “retro rate changes” when the original user has spent all their money)<o:p></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">Jim Lux<o:p></o:p></span></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div><div><div style="border-style: solid none none; border-top-color: rgb(225, 225, 225); border-top-width: 1pt; padding: 3pt 0in 0in;"><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><b><span style="font-size: 11pt; font-family: Calibri, sans-serif;">From:</span></b><span style="font-size: 11pt; font-family: Calibri, sans-serif;"><span class="Apple-converted-space"> </span>Beowulf [<a href="mailto:beowulf-bounces@beowulf.org">mailto:beowulf-bounces@beowulf.org</a>]<span class="Apple-converted-space"> </span><b>On Behalf Of<span class="Apple-converted-space"> </span></b>Olli-Pekka Lehto<br><b>Sent:</b><span class="Apple-converted-space"> </span>Monday, May 11, 2015 11:48 AM<br><b>To:</b><span class="Apple-converted-space"> </span>John Hearns<br><b>Cc:</b><span class="Apple-converted-space"> </span>Beowulf Mailing List<br><b>Subject:</b><span class="Apple-converted-space"> </span>Re: [Beowulf] HPC in the cloud question<o:p></o:p></span></div></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">We have a similar service intended especially for colocating the datacenters of Polytechnics and Universities in our datacenter in the north of Finland. <o:p></o:p></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><a href="http://www.slideshare.net/PeterJenkins1/csc-modular-datacenter" style="color: purple; text-decoration: underline;">http://www.slideshare.net/PeterJenkins1/csc-modular-datacenter</a><o:p></o:p></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">In addition we have been operating an HPC-oriented IaaS-cloud, carved off our production cluster for over a year now (<a href="https://research.csc.fi/cloud-computing" style="color: purple; text-decoration: underline;">https://research.csc.fi/cloud-computing</a>). One thing that’s under active development is a virtual cluster toolchain and front-end which could fairly easily be utilized by other sites as well: <a href="https://github.com/CSC-IT-Center-for-Science/pouta-blueprints" style="color: purple; text-decoration: underline;">https://github.com/CSC-IT-Center-for-Science/pouta-blueprints</a><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">Recently there’s been a growing demand for private cloud for internal projects and even from other public institutions. They present a possibility that the service may evolve to become a more general-purpose cloud platform that<span class="Apple-converted-space"> </span><i>also</i> supports HPC workloads. The marginal cost of this is fairly reasonable as much of the heavy lifting is in the cloud middleware development/integration that needs to be done anyway and adding different types of nodes/flavours is pretty trivial. <o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">This trend presents an interesting prospect for HPC centers in general: I’m willing to bet that in many places around the globe there is a niche for a vendor-independent, non-profit, regional, government-backed cloud service for critical public-sector workloads. HPC centers are be a good fit for providing this as many are already developing their own cloud services, procure and manage large quantities of scale-out hardware and have typically a very trustworthy reputation (and possibly certifications). <o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">Perhaps in the future the circle will close and we'll see some HPC centers become again providers of mission-critical general-puropse centralized computing resources in addition to HPC. :)<o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">O-P<o:p></o:p></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-family: Helvetica, sans-serif;">-- <br>Olli-Pekka Lehto<br>Development Manager, Computing Platforms <o:p></o:p></span></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-family: Helvetica, sans-serif;">CSC - IT Center for Science Ltd.<br>E-Mail: <a href="mailto:olli-pekka.lehto@csc.fi" style="color: purple; text-decoration: underline;">olli-pekka.lehto@csc.fi</a><span class="Apple-converted-space"> </span>// Tel: +358 50 381 8604 // skype: oplehto // twitter: @ople<o:p></o:p></span></div></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div><div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">On 10 May 2015, at 21:47, John Hearns <<a href="mailto:hearnsj@googlemail.com" style="color: purple; text-decoration: underline;">hearnsj@googlemail.com</a>> wrote:<o:p></o:p></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><br><br><o:p></o:p></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt;"><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">This article might be interesting:<o:p></o:p></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><a href="http://www.information-age.com/technology/data-centre-and-it-infrastructure/123459441/inside-uks-first-collaborative-data-centre" style="color: purple; text-decoration: underline;">http://www.information-age.com/technology/data-centre-and-it-infrastructure/123459441/inside-uks-first-collaborative-data-centre</a><o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">As it says 'Data-centre-as-a-service'<o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">A shared data centre, outside the centre of the city, used by several research inistitutes and universities.<o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">I have been involved in preparing bids for equipment there, including the innovative eMedlab project.<o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">Central London has its own problems in getting enough space and power for large computing setups, and this makes a lot of sense.<o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">On 8 May 2015 at 20:58, Dimitris Zilaskos <<a href="mailto:dimitrisz@gmail.com" target="_blank" style="color: purple; text-decoration: underline;">dimitrisz@gmail.com</a>> wrote:<o:p></o:p></div><blockquote style="border-style: none none none solid; border-left-color: rgb(204, 204, 204); border-left-width: 1pt; padding: 0in 0in 0in 6pt; margin-left: 4.8pt; margin-right: 0in;"><div><div><div><p class="MsoNormal" style="margin: 0in 0in 12pt; font-size: 12pt; font-family: 'Times New Roman', serif;">Hi,<br><br>IBM Platform does provide IB for HPC with bare metal and cloudbursting, among other HPC services on the cloud. Detailed information including benchmarks can be found at<span class="Apple-converted-space"> </span><a href="http://www-03.ibm.com/systems/platformcomputing/products/cloudservice/" target="_blank" style="color: purple; text-decoration: underline;">http://www-03.ibm.com/systems/platformcomputing/products/cloudservice/</a><span class="Apple-converted-space"> </span>. Note that I work for IBM so I am obviously biased.<o:p></o:p></p></div><p class="MsoNormal" style="margin: 0in 0in 12pt; font-size: 12pt; font-family: 'Times New Roman', serif;">Best regards,<o:p></o:p></p></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">Dimitris<o:p></o:p></div></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">On Fri, May 8, 2015 at 2:40 PM, Prentice Bisbal <<a href="mailto:prentice.bisbal@rutgers.edu" target="_blank" style="color: purple; text-decoration: underline;">prentice.bisbal@rutgers.edu</a>> wrote:<o:p></o:p></div><blockquote style="border-style: none none none solid; border-left-color: rgb(204, 204, 204); border-left-width: 1pt; padding: 0in 0in 0in 6pt; margin-left: 4.8pt; margin-right: 0in;"><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">Mike,<br><br>What are the characteristics of your cluster workloads? Are they tightly coupled jobs, or are they embarassingly parallel or serial jobs? I find it hard to believe that a virtualized, ethernet shared network infrastructure can compete with FDR IB for performance on tightly coupled jobs. AWS HPC representatives came to my school to give a presentation on their offerings, and even they admitted as much.<br><br>If your workloads are communication intensive, I'd think harder about using the cloud, or find a cloud provider that provides IB for HPC (there are a few that do, but I can't remember their names). If your workloads are loosely-coupled jobs or many serial jobs, AWS or similar might be fine. AWS does not provide IB, and in fact shares very little information about their network architecture, making it had to compare to other offerings without actually running benchmarks.<br><br>If your users primarily interact with the cluster through command-line logins, using the cloud shouldn't be noticeably different the hostname(s) they have to SSH to will be different, and moving data in an out might be different, but compiling and submitting jobs should be the same if you make the same tools available in the cloud that you have on your local clusters.<span style="color: rgb(136, 136, 136);"><br><br>Prentice</span><o:p></o:p></div><div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><br><br><br><br>On 05/07/2015 06:28 PM, Hutcheson, Mike wrote:<o:p></o:p></div><blockquote style="border-style: none none none solid; border-left-color: rgb(204, 204, 204); border-left-width: 1pt; padding: 0in 0in 0in 6pt; margin-left: 4.8pt; margin-right: 0in;"><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">Hi. We are working on refreshing the centralized HPC cluster resources<br>that our university researchers use. I have been asked by our<br>administration to look into HPC in the cloud offerings as a possibility to<br>purchasing or running a cluster on-site.<br><br>We currently run a 173-node, CentOS-based cluster with ~120TB (soon to<br>increase to 300+TB) in our datacenter. It¹s a standard cluster<br>configuration: IB network, distributed file system (BeeGFS. I really<br>like it), Torque/Maui batch. Our users run a varied workload, from<br>fine-grained, MPI-based parallel aps scaling to 100s of cores to<br>coarse-grained, high-throughput jobs (We¹re a CMS Tier-3 site) with high<br>I/O requirements.<br><br>Whatever we transition to, whether it be a new in-house cluster or<br>something ³out there², I want to minimize the amount of change or learning<br>curve our users would have to experience. They should be able to focus on<br>their research and not have to spend a lot of their time learning a new<br>system or trying to spin one up each time they have a job to run.<br><br>If you have worked with HPC in the cloud, either as an admin and/or<br>someone who has used cloud resources for research computing purposes, I<br>would appreciate learning your experience.<br><br>Even if you haven¹t used the cloud for HPC computing, please feel free to<br>share your thoughts or concerns on the matter.<br><br>Sort of along those same lines, what are your thoughts about leasing a<br>cluster and running it on-site?<br><br>Thanks for your time,<br><br>Mike Hutcheson<br>Assistant Director of Academic and Research Computing Services<br>Baylor University<br><br><br>_______________________________________________<br>Beowulf mailing list,<span class="Apple-converted-space"> </span><a href="mailto:Beowulf@beowulf.org" target="_blank" style="color: purple; text-decoration: underline;">Beowulf@beowulf.org</a><span class="Apple-converted-space"> </span>sponsored by Penguin Computing<br>To change your subscription (digest mode or unsubscribe) visit<span class="Apple-converted-space"> </span><a href="http://www.beowulf.org/mailman/listinfo/beowulf" target="_blank" style="color: purple; text-decoration: underline;">http://www.beowulf.org/mailman/listinfo/beowulf</a><o:p></o:p></div></blockquote><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><br>_______________________________________________<br>Beowulf mailing list,<span class="Apple-converted-space"> </span><a href="mailto:Beowulf@beowulf.org" target="_blank" style="color: purple; text-decoration: underline;">Beowulf@beowulf.org</a><span class="Apple-converted-space"> </span>sponsored by Penguin Computing<br>To change your subscription (digest mode or unsubscribe) visit<span class="Apple-converted-space"> </span><a href="http://www.beowulf.org/mailman/listinfo/beowulf" target="_blank" style="color: purple; text-decoration: underline;">http://www.beowulf.org/mailman/listinfo/beowulf</a><o:p></o:p></div></div></blockquote></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div></div><p class="MsoNormal" style="margin: 0in 0in 12pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><br>_______________________________________________<br>Beowulf mailing list,<span class="Apple-converted-space"> </span><a href="mailto:Beowulf@beowulf.org" style="color: purple; text-decoration: underline;">Beowulf@beowulf.org</a><span class="Apple-converted-space"> </span>sponsored by Penguin Computing<br>To change your subscription (digest mode or unsubscribe) visit<span class="Apple-converted-space"> </span><a href="http://www.beowulf.org/mailman/listinfo/beowulf" target="_blank" style="color: purple; text-decoration: underline;">http://www.beowulf.org/mailman/listinfo/beowulf</a><o:p></o:p></p></blockquote></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">_______________________________________________<br>Beowulf mailing list,<span class="Apple-converted-space"> </span><a href="mailto:Beowulf@beowulf.org" style="color: purple; text-decoration: underline;">Beowulf@beowulf.org</a><span class="Apple-converted-space"> </span>sponsored by Penguin Computing<br>To change your subscription (digest mode or unsubscribe) visit<span class="Apple-converted-space"> </span><a href="http://www.beowulf.org/mailman/listinfo/beowulf" style="color: purple; text-decoration: underline;">http://www.beowulf.org/mailman/listinfo/beowulf</a><o:p></o:p></div></blockquote></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><o:p> </o:p></div></div></div></div>_______________________________________________<br>Beowulf mailing list,<span class="Apple-converted-space"> </span><a href="mailto:Beowulf@beowulf.org" style="color: purple; text-decoration: underline;">Beowulf@beowulf.org</a><span class="Apple-converted-space"> </span>sponsored by Penguin Computing<br>To change your subscription (digest mode or unsubscribe) visit<span class="Apple-converted-space"> </span><a href="http://www.beowulf.org/mailman/listinfo/beowulf" style="color: purple; text-decoration: underline;">http://www.beowulf.org/mailman/listinfo/beowulf</a></div></blockquote></div><br></div></div></body></html>