[Beowulf] Gentoo in the HPC environment
landman at scalableinformatics.com
Mon Jun 30 10:00:37 PDT 2014
On 06/30/2014 12:42 PM, rf at q-leap.de wrote:
>>>>>> "Joe" == Joe Landman <landman at scalableinformatics.com> writes:
> Joe> On 06/30/2014 11:27 AM, Prentice Bisbal wrote:
> >> I second Gavin.
> Prentice> A lot of people have been mentioning LXC and Docker ans
> Prentice> cures to this problem, and to paraphrase The Princess
> Prentice> Bride, you keep using those words I don't think they mean
> Prentice> what you think they mean. Docker and LXC are great for
> Prentice> isolating running services: apache, DNS, etc. For the most
> Prentice> part, we are stalking about user-space libraries and
> Prentice> programs. I don't see how Docker and LXC could be used or
> Prentice> provide any benefit in this context.
> Joe> We can create a completely repeatable portable mechanism to
> Joe> distribute applications with full dependency chains as part of
> Joe> the distribution, across machines of any linux distro type,
> Joe> without impact core packages (which in the case of specific
> Joe> distros are often non-functional for anything but legacy system
> Joe> work) ... and you don't see the benefit to this?
> Joe> Seriously?
> Joe> Quick show of hands: Anyone running an HPC system, ever run
> Joe> into, say, a dependency hell/nightmare due to a package
> Joe> requirement?
> I think your overemphasizing the upside of this approach. Sure, if you
> have 2-3 apps like this, it's still feasible to manage. If it becomes a
> lot more than that (and in a larger compute center it would), you
> essentially have to manage Docker instances like OS installations (minus
> kernel). Do you really want to do that for more than a couple of them?
As with any technology, there is a cost and a benefit. Moreover, there
are no silver bullets, unicorns, or any other magical incantation that
will make bad things good, etc.
One must weigh the costs against the benefits. Part of the costs are
more vigilance required in security contexts. Part of the benefits are
much simpler deployment/management.
> You might say: Well the software vendors are going to supply and manage
> the Docker instances. Will you trust them? I'd say: Welcome to the Android app
Well, no, I wouldn't say that. I would imagine each center would create
their own containers, and mange them. Or supply a container
build/testing environment to their users for them to build their own for
This is why in part, the market for pre-build VMs is effectively non
existent, yet everyone wants to roll their own cloud/VMs. Same reason.
Provide the tech and get out of the way.
> world, trojans, backdoors, other security holes. And I'm not really
> convinced the container isolation is always going to protect us from this.
> I believe nobody wants this in their data center.
Same issues exist at the OS level. Containerization is a weaker form of
isolation than a VM. It has benefits, it has risks. You can crash a VM
without taking down the host. You can't crash a container without
requiring a reboot of the host. Risk is higher, but for a well behaved
app ... most are ... this shouldn't be a problem.
> Don't get me wrong. I also find the Docker concept appealing at first
> sight. But I somehow see a security and/or manageability nightmare wave
> coming up upon us with it ...
I am not convinced that this is as much of an issue as you think on the
manageability side. The security side is an issue for apps in general.
But then again, its not that much different than having any sort of
access to /dev/[k]mem, etc. Bad things can and do happen from good
apps, and malicious apps as well.
Docker and its ilk cannot protect you from malicious apps. kvm can
isolate a VM to contain damage. Intelligent policy, alerting, etc. and
sane backup/snapshots are a significant line of defense.
Prentiss opined that he didn't find Docker a beneficial concept as
compared to others. I (strongly) disagree with this. You opine that
security and other issues exist. I do agree with this. But its
non-sequitur as these issues exist independent of Docker/containers, and
Docker/containers and kvm for that matter, do not mask off these issues.
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
email: landman at scalableinformatics.com
web : http://scalableinformatics.com
twtr : @scalableinfo
phone: +1 734 786 8423 x121
cell : +1 734 612 4615
More information about the Beowulf