[Beowulf] Containers in HPC

[Bill Broadley]

On 5/23/19 3:49 AM, Jonathan Aquilina wrote:> Hi Guys,
>
>
>
> Can  someone clarify for me are containers another form of virtualized systems?
> Or are they isolated environments running on bare metal?

Generally virtual machines run their own kernel.  Typically CPU overhead is
close to zero, but things like network or disk I/O can be heavily impacted.  VMs
also typically require carving out a chunk of ram from the host system and
giving it to the guest.  So the memory overhead is inflexible, and mostly
static.  There are workarounds (like balloon memory drivers), but generally the
memory overhead is high.  Virtual machines also boot much like a regular OS, 10s
of seconds to minutes is common.

Containers do not involve a second kernel, but instead use cgroups (or similar
on other platforms) to give a container a chunk of system resources.  This makes
it easy to run a container expecting a different set of libraries, file system
layout, accounts, namespace, filesystems, etc to run on the same host.  While
you can limit the ram allocated to a container, it only has to consume what it
needs.  Cgroups can limit what a container can do, but generally the isolation
is not as good as with a virtual machine.  Containers can launch in a small
fraction of a second.  One experiment I did ran fedora, rhel, and ubuntu
containers and ran "uname -a" or equivalent in all 3.  I was able to launch all
3, get the output, and shut them all down in under 1 second.

The I/O and network overhead of containers is minimal, because you are using the
same kernel.  To the host kernel the difference between a container and a
process is minimal.

To further confuse things, often people end up running a collection of
containers in a virtual machine.  Kubernetes (and many other platforms) can use
this model.  But you can run containers on "bare metal", without using any
virtual machine, just directly on the underlying OS.

Hopefully that helps.