[Beowulf] Gentoo in the HPC environment

[Joe Landman]

On 06/30/2014 12:42 PM, rf at q-leap.de wrote:
>>>>>> "Joe" == Joe Landman <landman at scalableinformatics.com> writes:
>
>      Joe> On 06/30/2014 11:27 AM, Prentice Bisbal wrote:
>      >> I second Gavin.
>      >>
>      Prentice> A lot of people have been mentioning LXC and Docker ans
>      Prentice> cures to this problem, and to paraphrase The Princess
>      Prentice> Bride, you keep using those words I don't think they mean
>      Prentice> what you think they mean. Docker and LXC are great for
>      Prentice> isolating running services: apache, DNS, etc. For the most
>      Prentice> part, we are stalking about user-space libraries and
>      Prentice> programs. I don't see how Docker and LXC could be used or
>      Prentice> provide any benefit in this context.
>
>      Joe> We can create a completely repeatable portable mechanism to
>      Joe> distribute applications with full dependency chains as part of
>      Joe> the distribution, across machines of any linux distro type,
>      Joe> without impact core packages (which in the case of specific
>      Joe> distros are often non-functional for anything but legacy system
>      Joe> work) ... and you don't see the benefit to this?
>
>      Joe> Seriously?
>
>      Joe> Quick show of hands: Anyone running an HPC system, ever run
>      Joe> into, say, a dependency hell/nightmare due to a package
>      Joe> requirement?
>
> I think your overemphasizing the upside of this approach. Sure, if you
> have 2-3 apps like this, it's still feasible to manage. If it becomes a
> lot more than that (and in a larger compute center it would), you
> essentially have to manage Docker instances like OS installations (minus
> kernel). Do you really want to do that for more than a couple of them?

As with any technology, there is a cost and a benefit.  Moreover, there 
are no silver bullets, unicorns, or any other magical incantation that 
will make bad things good, etc.

One must weigh the costs against the benefits.  Part of the costs are 
more vigilance required in security contexts.  Part of the benefits are 
much simpler deployment/management.

> You might say: Well the software vendors are going to supply and manage
> the Docker instances. Will you trust them? I'd say: Welcome to the Android app

Well, no, I wouldn't say that.  I would imagine each center would create 
their own containers, and mange them.  Or supply a container 
build/testing environment to their users for them to build their own for 
active deployment.

This is why in part, the market for pre-build VMs is effectively non 
existent, yet everyone wants to roll their own cloud/VMs.  Same reason. 
  Provide the tech and get out of the way.

> world, trojans, backdoors, other security holes. And I'm not really
> convinced the container isolation is always going to protect us from this.
> I believe nobody wants this in their data center.

Same issues exist at the OS level.  Containerization is a weaker form of 
isolation than a VM.  It has benefits, it has risks.  You can crash a VM 
without taking down the host.  You can't crash a container without 
requiring a reboot of the host.  Risk is higher, but for a well behaved 
app ... most are ... this shouldn't be a problem.

> Don't get me wrong. I also find the Docker concept appealing at first
> sight. But I somehow see a security and/or manageability nightmare wave
> coming up upon us with it ...

I am not convinced that this is as much of an issue as you think on the 
manageability side.  The security side is an issue for apps in general.

But then again, its not that much different than having any sort of 
access to /dev/[k]mem, etc.  Bad things can and do happen from good 
apps, and malicious apps as well.

Docker and its ilk cannot protect you from malicious apps.  kvm can 
isolate a VM to contain damage.  Intelligent policy, alerting, etc. and 
sane backup/snapshots are a significant line of defense.

C.f. http://www.theregister.co.uk/2014/06/19/docker_security/


Prentiss opined that he didn't find Docker a beneficial concept as 
compared to others.  I (strongly) disagree with this.  You opine that 
security and other issues exist.  I do agree with this.  But its 
non-sequitur as these issues exist independent of Docker/containers, and 
Docker/containers and kvm for that matter, do not mask off these issues.



-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
twtr : @scalableinfo
phone: +1 734 786 8423 x121
cell : +1 734 612 4615