[Beowulf] Gentoo in the HPC environment

[Joe Landman]

Instead of letting this devolve into a distro battle (I have no dog in 
that race, but I know from long hard experience what to avoid), it makes 
more sense to look at the bigger picture.

In the larger frame, a cluster is a mechanism to provide computing 
cycles.  The keepers of the cluster are service folks, in the sense that 
they are providing a shared resource with specific functionality, and 
providing a service to the internal (and sometimes external) consumers 
of the service.

In this day and age of software defined everything, a cluster needs to 
be as flexible as possible, and provide the necessary level and type of 
service to be viable.  Not simply economically viable, but practical, 
and pragmatic.

Which means cluster admins and service teams need to address many 
different environmental issues and requests.

In academic circles, where there may be less of a push for commercial 
support on software, these requirements may be relaxed relative to other 
users.

In commercial circles, where one might need to guarantee results (for 
any number of reasons, and yes, this happens), the environments are far 
more rigid.

How can a provider of cycles provide service to a rigid set of 
requirements without being flexible?

My argument is, fundamentally, that technologies like kvm, and Docker on 
Linux provide a simple mechanism for that functionality.  On Windows 
(very few windows clusters, but still) you can do this with HyperV.

So the details of what runs at the base level on the cluster matter far 
less than the detailed requirements and the business needs for the 
application.  The latter should determine the former, and if the latter 
requires something different than the former supplies, kvm/Docker etc. 
can provide this.  So can bare metal stateless.

Or conversely, you could simply provide exactly one type of computing, 
and watch your users go elsewhere, specifically to resources that will 
give them what they require.  Somehow that seems to be not-precisely 
what this crowd would want though.

Its just a thought though.  Gentoo or not doesn't matter as much as 
*how* your users need to use it.  Thats the point of pain.  If the 
distro can't handle it, or isn't supported correctly, yes, you'll need 
to change.  If your cycle provider is rigid in what they will provide, 
its pretty easy to go to another cycle provider.

This is what clusters in clouds have created.  This is why there are 
folks like Cycle Computing for cloud based clusters, and many good folks 
like Sabalcore with bare metal systems.  Application and business needs 
dictate platform choices.

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
twtr : @scalableinfo
phone: +1 734 786 8423 x121
cell : +1 734 612 4615