<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi,</p>
<p>In the same vein, Benoit (cc of this mail) created an automatic
way to install a whole cluster based on centos/salt/slurm... Take
a look here : <a class="moz-txt-link-freetext" href="https://github.com/oxedions/banquise">https://github.com/oxedions/banquise</a><br>
</p>
<p>On my side, I play with LXD to automate all this stuff (but I
thought about doing it with singularity too). The main idea is to
have a stable OS on the hardware that you do not need to
(re-)install anymore but just some containers that you can move or
resize to fit your needs. I do not know how exactly, but IMHO, I
think our work should move to this kind of DevOps things as in the
cloud area. Actually I have some salt recipies to orchestrate
hardware reinstallation for some nodes (not all my clusters), and
then, I apply automatically other formulas to deploy the
containers based on some specifications.<br>
</p>
<p><br>
</p>
<p>Best regards,</p>
<p>Remy<br>
</p>
<p><br>
</p>
<br>
<div class="moz-cite-prefix">Le 29/09/2016 à 13:33, Olli-Pekka Lehto
a écrit :<br>
</div>
<blockquote
cite="mid:1896339090.747267.1475148833503.JavaMail.zimbra@csc.fi"
type="cite">
<div style="font-family: arial, helvetica, sans-serif; font-size:
12pt; color: #000000">
<div>We have our latest cluster software stack for a distributed
set of clusters built on Ansible:</div>
<div><a class="moz-txt-link-freetext" href="https://github.com/CSC-IT-Center-for-Science/fgci-ansible">https://github.com/CSC-IT-Center-for-Science/fgci-ansible</a></div>
<div><br data-mce-bogus="1">
</div>
<div>
<div>A recent presentation at the SLURM User Group on
Ansiblizing SLURM: </div>
<div><a class="moz-txt-link-freetext" href="https://gitpitch.com/CSC-IT-Center-for-Science/ansible-role-slurm/gitpitch">https://gitpitch.com/CSC-IT-Center-for-Science/ansible-role-slurm/gitpitch</a></div>
</div>
<div><br data-mce-bogus="1">
</div>
<div>I see benefits also in being able to share playbooks and
collaborate on improving them with other teams in our
organization and the Universities, even ones working in
non-HPC areas. </div>
<div><br>
</div>
<div>Best regards,</div>
<div>Olli-Pekka</div>
<div data-marker="__SIG_PRE__">-- <br>
Olli-Pekka Lehto<br>
Development Manager<br>
Computing Platforms<br>
CSC - IT Center for Science Ltd.<br>
E-Mail: <a class="moz-txt-link-abbreviated" href="mailto:olli-pekka.lehto@csc.fi">olli-pekka.lehto@csc.fi</a><br>
Tel: +358 50 381 8604<br>
skype: oplehto // twitter: ople</div>
<br>
<hr id="zwchr" data-marker="__DIVIDER__">
<div data-marker="__HEADERS__">
<blockquote style="border-left: 2px solid #1010FF;
margin-left: 5px; padding-left: 5px; color: #000;
font-weight: normal; font-style: normal; text-decoration:
none; font-family: Helvetica,Arial,sans-serif; font-size:
12pt;" data-mce-style="border-left: 2px solid #1010FF;
margin-left: 5px; padding-left: 5px; color: #000;
font-weight: normal; font-style: normal; text-decoration:
none; font-family: Helvetica,Arial,sans-serif; font-size:
12pt;"><b>From: </b>"Craig Andrew"
<a class="moz-txt-link-rfc2396E" href="mailto:cbandrew@wi.mit.edu"><cbandrew@wi.mit.edu></a><br>
<b>To: </b>"Tim Cutts" <a class="moz-txt-link-rfc2396E" href="mailto:tjrc@sanger.ac.uk"><tjrc@sanger.ac.uk></a><br>
<b>Cc: </b><a class="moz-txt-link-abbreviated" href="mailto:beowulf@beowulf.org">beowulf@beowulf.org</a><br>
<b>Sent: </b>Wednesday, 28 September, 2016 18:01:59<br>
<b>Subject: </b>Re: [Beowulf] more automatic building<br>
</blockquote>
</div>
<div data-marker="__QUOTED_TEXT__">
<blockquote style="border-left: 2px solid #1010FF;
margin-left: 5px; padding-left: 5px; color: #000;
font-weight: normal; font-style: normal; text-decoration:
none; font-family: Helvetica,Arial,sans-serif; font-size:
12pt;" data-mce-style="border-left: 2px solid #1010FF;
margin-left: 5px; padding-left: 5px; color: #000;
font-weight: normal; font-style: normal; text-decoration:
none; font-family: Helvetica,Arial,sans-serif; font-size:
12pt;">
<div style="font-family: arial,helvetica,sans-serif;
font-size: 12pt; color: #000000;"
data-mce-style="font-family: arial,helvetica,sans-serif;
font-size: 12pt; color: #000000;">
<div>I agree with Tim. </div>
<br>
<div>We are finishing up an Ansible install and it has
worked well for us. </div>
<br>
<div>Initially, we used it internally to help standardize
our cluster builds, but is has many more uses. We
recently used it to provision a VM that we saved off and
uploaded to Amazon for building an AMI. You can also use
it to change attributes on your running systems. I have
used at Cobler in the past and it works well, too. I
just find Ansible to be a little easier.</div>
<div><br>
Good luck,</div>
<div>Craig</div>
<br>
<div>Craig Andrew<br>
Manager of Systems Administration</div>
<div>Whitehead Institute for Biomedical Research</div>
<br>
<hr id="zwchr">
<div><b>From: </b>"Tim Cutts" <a class="moz-txt-link-rfc2396E" href="mailto:tjrc@sanger.ac.uk"><tjrc@sanger.ac.uk></a><br>
<b>To: </b>"Mikhail Kuzminsky" <a class="moz-txt-link-rfc2396E" href="mailto:mikky_m@mail.ru"><mikky_m@mail.ru></a>,
<a class="moz-txt-link-abbreviated" href="mailto:beowulf@beowulf.org">beowulf@beowulf.org</a><br>
<b>Sent: </b>Wednesday, September 28, 2016 10:46:41 AM<br>
<b>Subject: </b>Re: [Beowulf] more automatic building<br>
</div>
<br>
<div>
<style><!--
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:Calibri;
font-variant:normal !important;
color:windowtext;
text-transform:none;
mso-contextual-alternates:no;
font-weight:normal;
font-style:normal;
text-decoration:none none;
vertical-align:baseline;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:595.0pt 842.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size: 11.0pt;
font-family: Calibri; mso-fareast-language:
EN-US;" data-mce-style="font-size: 11.0pt;
font-family: Calibri; mso-fareast-language:
EN-US;">Any number of approaches will work. When
I used to do this years ago (I've long since
passed on the technical side) I'd PXE boot,
partition the hard disk and set up a provisioning
network and base OS install using the Debian FAI
(Fully Automated Install) system, and then use
cfengine to configure the machine once it had come
in that minimal state. This approach was used
across the board for all of our Linux boxes, from
Linux desktops to database servers to HPC compute
nodes.</span></p>
<p class="MsoNormal"><span style="font-size: 11.0pt;
font-family: Calibri; mso-fareast-language:
EN-US;" data-mce-style="font-size: 11.0pt;
font-family: Calibri; mso-fareast-language:
EN-US;"> </span></p>
<p class="MsoNormal"><span style="font-size: 11.0pt;
font-family: Calibri; mso-fareast-language:
EN-US;" data-mce-style="font-size: 11.0pt;
font-family: Calibri; mso-fareast-language:
EN-US;">These days the team uses tools like
cobbler and ansible to achieve the same thing.
There are lots of ways to do it, but the principle
is the same.</span></p>
<p class="MsoNormal"><span style="font-size: 11.0pt;
font-family: Calibri; mso-fareast-language:
EN-US;" data-mce-style="font-size: 11.0pt;
font-family: Calibri; mso-fareast-language:
EN-US;"> </span></p>
<p class="MsoNormal"><span style="font-size: 11.0pt;
font-family: Calibri; mso-fareast-language:
EN-US;" data-mce-style="font-size: 11.0pt;
font-family: Calibri; mso-fareast-language:
EN-US;">Tim</span></p>
<p class="MsoNormal"><span style="font-size: 11.0pt;
font-family: Calibri; mso-fareast-language:
EN-US;" data-mce-style="font-size: 11.0pt;
font-family: Calibri; mso-fareast-language:
EN-US;"> </span></p>
<div>
<p class="MsoNormal"><span style="font-size: 10.5pt;
font-family: Helvetica; color: black;"
data-mce-style="font-size: 10.5pt; font-family:
Helvetica; color: black;">-- </span></p>
<div>
<p class="MsoNormal"><span style="font-size:
10.5pt; font-family: Helvetica; color: black;"
data-mce-style="font-size: 10.5pt;
font-family: Helvetica; color: black;">Head of
Scientific Computing</span></p>
</div>
</div>
<p class="MsoNormal"><span style="font-size: 10.5pt;
font-family: Helvetica; color: black;
mso-fareast-language: EN-US;"
data-mce-style="font-size: 10.5pt; font-family:
Helvetica; color: black; mso-fareast-language:
EN-US;">Wellcome Trust Sanger Institute</span><span
style="font-size: 11.0pt; font-family: Calibri;
mso-fareast-language: EN-US;"
data-mce-style="font-size: 11.0pt; font-family:
Calibri; mso-fareast-language: EN-US;"></span></p>
<p class="MsoNormal"><span style="font-size: 11.0pt;
font-family: Calibri; mso-fareast-language:
EN-US;" data-mce-style="font-size: 11.0pt;
font-family: Calibri; mso-fareast-language:
EN-US;"> </span></p>
<div>
<div>
<p class="MsoNormal">On 28/09/2016, 15:34,
"Beowulf on behalf of Mikhail Kuzminsky" <<a
moz-do-not-send="true"
href="mailto:beowulf-bounces@beowulf.org"
target="_blank">beowulf-bounces@beowulf.org</a>
on behalf of
<a moz-do-not-send="true"
href="mailto:mikky_m@mail.ru" target="_blank">mikky_m@mail.ru</a>>
wrote:</p>
</div>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<blockquote style="border: none; border-left: solid
#B5C4DF 4.5pt; padding: 0cm 0cm 0cm 4.0pt;
margin-left: 3.75pt; margin-right: 0cm;"
id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE"
data-mce-style="border: none; border-left: solid
#B5C4DF 4.5pt; padding: 0cm 0cm 0cm 4.0pt;
margin-left: 3.75pt; margin-right: 0cm;">
<div>
<div>
<p class="MsoNormal" style="margin-bottom:
12.0pt;" data-mce-style="margin-bottom:
12.0pt;">I worked always w/very small HPC
clusters and built them manually (each
server).
<br>
But what is reasonable to do for clusters
containing some tens or hundred of nodes ?<br>
Of course w/modern Xeon (or Xeon Phi KNL) and
IB EDR, during the next year for example.
<br>
There are some automatic systems like OSCAR or
even ROCKS. <br>
<br>
But it looks that ROCKS don't support modern
interconnects, and there may be problems
<br>
w/OSCAR versions for support of systemd-based
distributives like CentOS 7. For next year -<br>
is it reasonable to wait new OSCAR version or
something else ?<br>
<br>
Mikhail Kuzminsky,<br>
Zelinsky Institute of Organic Chemistry RAS,<br>
Moscow <br>
<br>
<br>
</p>
</div>
</div>
</blockquote>
</div>
-- The Wellcome Trust Sanger Institute is operated by
Genome Research Limited, a charity registered in England
with number 1021457 and a company registered in England
with number 2742969, whose registered office is 215
Euston Road, London, NW1 2BE. <br>
_______________________________________________<br>
Beowulf mailing list, <a class="moz-txt-link-abbreviated" href="mailto:Beowulf@beowulf.org">Beowulf@beowulf.org</a> sponsored by
Penguin Computing<br>
To change your subscription (digest mode or unsubscribe)
visit <a class="moz-txt-link-freetext" href="http://www.beowulf.org/mailman/listinfo/beowulf">http://www.beowulf.org/mailman/listinfo/beowulf</a><br>
</div>
<br>
<div>-- <br>
</div>
<div>___________________________________________<br>
<br>
<br>
</div>
</div>
<br>
_______________________________________________<br>
Beowulf mailing list, <a class="moz-txt-link-abbreviated" href="mailto:Beowulf@beowulf.org">Beowulf@beowulf.org</a> sponsored by
Penguin Computing<br>
To change your subscription (digest mode or unsubscribe)
visit <a class="moz-txt-link-freetext" href="http://www.beowulf.org/mailman/listinfo/beowulf">http://www.beowulf.org/mailman/listinfo/beowulf</a><br>
</blockquote>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Beowulf mailing list, <a class="moz-txt-link-abbreviated" href="mailto:Beowulf@beowulf.org">Beowulf@beowulf.org</a> sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit <a class="moz-txt-link-freetext" href="http://www.beowulf.org/mailman/listinfo/beowulf">http://www.beowulf.org/mailman/listinfo/beowulf</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Rémy Dernat
Ingénieur d'Etudes
MBB/ISE-M</pre>
</body>
</html>