A webinar presented for the {code} Community on August 30, 2017. In this talk, we looked at the sphere of modern container runtimes that start with Docker's emergence in 2013/2014 to today's additions of rkt, OCI's runc, containerd, cri-o, and Cloud Foundry's garden-runc project, many of them consolidating around the OCI standard for container runtime and image specifications.

1. Container
Runtimes:
Comparing and
contrasting
today’s engines

2. WHO AM I?
Phil Estes
Senior Technical Staff Member
Office of the CTO, IBM Watson & Cloud Platform
Maintainer, Docker engine
Maintainer, containerd
Contributor, OCI/runc
Docker Captain & {code} Catalyst
Blog: https://integratedcode.us
Twitter: @estesp

3. BORING!!
Let’s Make Containers Boring - Vincent
Batts, Red Hat (meetup talk)
An Ode To Boring: Creating Open and Stable
Container World - Bob Wise (Medium)
The goal of standardising containers is,
ultimately, to make them boring - Jonathan
Boulle (Container Camp interview)
But many platform builders and operators are looking for “boring
infrastructure”: a basic component that provides the robust primitives
for running containers on their system, bundled in a stable interface,
and nothing else. - Docker Blog, containerd announcement

4. NOT
QUITE
AS
BORING

5. 1.
What is a
container
runtime?
Phil’s Dictionary Definition:
A software interface to operating
system “container” isolation
technology used to execute
lifecycle commands (create, start,
pause, resume, stop, delete)
against a container instance.

6. Unix/chroot
BSD
1982 2000 Today2008200720062005
FreeBSD
jails/Solaris zones
OpenVZ
Parallels
Cgroups/Process
Containers
IBM/Google
AIX Wpars
IBM
LXC
CONTAINER HISTORY LESSON

7. WHERE WE ARE TODAY IS MOSTLY DUE TO DOCKER
▸ 2013-2014: A better UX on top of LXC + image library
▸ 2014: libcontainer project (moving away from LXC)
▸ 2016-17: OCI, runc, containerd refactoring
Docker’s surge in popularity created an
environment where containers went from
relatively obscure Linux kernel technology to
standard developer tool for packaging software.

8. Runtime specification
Image specification
runC implementation
2013 2014 2015 2016 2017
Garden-runC
Guardian project
K8sCRI
*[0.2.x branch]
*[1.0 branch]

9. 2.
Comparing the
container
runtimes

10. libnetwork
VolumeAPI
AuthZ
ctr-shim
runc
{
/usr/bin/docker
/usr/bin/dockerd
/usr/bin/dockerd
/usr/bin/docker
Single statically-linked
binary comprised:
> Client
> Daemon
> Build tool
> Registry client
HTTP/JSON
Docker API
gRPC
API

11. ▸ Launched in December 2014
▸ Differences of opinion and direction caused CoreOS to
create rkt as a Docker alternative
▸ CoreOS created standards that rkt was an
implementation for: the appc spec comprised a
runtime configuration and the ACI image format
▸ Rkt moved to a “pod-native” approach to
containers/applications early on (v0.5.3)
▸ Daemon-less operation supported by systemd and,
specifically systemd-nspawn
▸ Contributed as a CNCF project in March 2017
Created in December 2014
> 63 releases (1.28.1 current)
> 185 contributors
> CoreOS created; now a CNCF project
> Available as Kubernetes CRI
Implementation, Mesos, Nomad drivers

12. ▸ Uses a stage execution model
to allow for different “stage1”
implementations:
▹ Qemu-kvm
▹ Systemd/nspawn
▹ “Fly” (simple chroot)
▹ 3rd party
implementations
▸ Appc Spec development
deprecated in favor of using
OCI specs for future rkt
releases.
Design/Implementation

13. & runC
> Announced June 20th, 2015
> Charter signed on
December 8th, 2015
> 44 member companies
> Both specifications
reached 1.0 last month
https://opencontainers.org
https://github.com/opencontainers
> runc is a client wrapper around libcontainer
> libcontainer is the OS level interface for containers
> OCI spec covers Solaris, Linux, & MS Windows
$ docker run -it --read-only
-v /host:/hostpath
alpine sh
/#
{
"ociVersion": "1.0.0",
"platform": {
"os": "linux",
"arch": "amd64"
},
"process": {
"terminal": true,
"args": [
"sh"
],
"env": [
"PATH=/usr/sbin:/usr/local/bin:/bin”
config.json
• A Linux Foundation Collaborative Project
• Free from control by any particular vendor’s specific cloud stack or ecosystem
• Includes a specification, reference runtime* and now, a specified image format
*seeded with runc + libcontainer by Docker

14. runC Created in June 2015
> 16 releases (1.0.0-rc4 current)
> 215 contributors
> OCI maintained/governance
> Used by Docker, containerd,
garden-runc/Guardian, others
▸ Runc is a client wrapper around the pre-existing
libcontainer library project
▸ Runc is one implementation of the OCI runtime
specification
▸ Scope of runc is clearly limited by OCI charter: no
networking, image handling/resolution, storage
support
▸ Enablement of low-level OS features happen here:
ambient caps, rootless containers, new cgroup support,
and so on
▸ Daemon-less operation; wrapping code must handle
any broader node and cluster level container mgmt.

15. Garden-runC/Guardian
Created in June 2015
> 29 releases (1.9.2 current)
> 40+ contributors
> CF project governance
> Garden CF runtime uses this
implementation to run containers
▸ Cloud Foundry is a enterprise-class PaaS open source
project that has used Linux containers for many years
▸ CF Garden-linux driver now is deprecated in favor of the
garden-runc/guardian codebase, using OCI+runc to
execute containers
▸ The guardian codebase effectively wraps runc with
network, image, volume, and rootfs management
support that runC doesn’t provide on its own
▸ Because of the use of OCI & runc, this guardian layer
has added support for interesting things like rootless
containers (experimental) and Windows container
support
https://github.com/cloudfoundry/garden-runc-release | https://github.com/cloudfoundry/guardian

16. Created in December 2015
> 22 releases (1.0.0-alpha6 current)
> 96 contributors
> Docker created; now a CNCF project
> Used by Docker, cri-containerd
(incubation project in K8s), AWS, VMWare
▸ Launched initially in December 2015 (used as part of a
Docker release in early 2016)
▸ Two streams of activity to discuss:
▹ “0.2.x” branch: used in today’s Docker releases as a
simple runc manager
▹ “1.0.0” branch: based on the December 2016
announcement and contribution of containerd to
CNCF for use as a core embeddable container
runtime for Kubernetes and other projects
▸ Executes containers using the OCI runc executor;
containerd manages state/metadata, image & registry
interactions, snapshot drivers (overlay, btrfs)
▸ Supports Windows, Linux, Solaris, multi-arch

17. Metadata Content Snapshotter
Runtime
Linux (shim)
OCI runC
IMAGE TASK CONTAINER
Client library (Golang)gRPC
Service
APIs
Vendor client library to embed containerd{ or } ▸ Metrics API &
Prometheus
support
▸ OCI runtime and
image support
▸ Clean API and
abstractions
▸ Pluggable runtime
support (used by
VMWare impl.)
▸ Namespace
support
(multi-tenancy)

18. Created in September 2016
> 6 releases (1.0.0-beta.0 current)
> 49 contributors
> Kubernetes incubation project
> Specifically created to implement
the K8s CRI; no standalone usage
▸ Launched in September 2016
▸ An assembly of components to implement the
Kubernetes CRI:
▹ OCI runc (or any OCI runtime implementation)
▹ containers/image & containers/storage GitHub
libraries
▹ Network support via CNI plugins
▸ Red Hat created and maintained project; promoted as
runtime for Kubernetes and OpenShift (K8s-based)
▸ Based on the idea that K8s has simple containerizer
needs: cri-o is “just enough” runtime built from OCI and
glue pieces to provide that baseline functionality

19. 3.
Summary

20. ▸ THE GOOD:
▹ Docker and an early shared community brought about a tidal
wave of interest in (existing) container technology
▹ Plenty of cross-industry focus today on stable, “boring”
container runtime technology; higher layers are well
supported and have choice in runtime implementations
▹ OCI brought a strong community of experts together to
create a well-defined specification to standardize containers
▹ OCI specs & implementations are being shared!
▸ THE NOT SO GOOD:
▹ Human nature: we like doing our own thing; some amount of
confusion still in our ecosystem re: runtime strategy and
what is politics versus technical differentiation
▹ Open source politics & Twitter battles are wasting time and
energy; especially time that could == innovative/creative
power of competitive vendors working together on stable
underpinnings

21. @estesp
github.com/estesp
estesp@gmail.com
https://integratedcode.us
IRC: estesp
THANK YOU!

22. CREDITS
Special thanks to all the people who made and released these
awesome resources for free:
▸ Simple line icons by Mirko Monti
▸ E-commerce icons by Virgil Pana
▸ Streamline iconset by Webalys
▸ Presentation template by SlidesCarnival