The document discusses the state of container security. It notes that while container security has improved significantly from early Docker implementations, there are still challenges. Container isolation features have known vulnerabilities and lightweight virtualization is gaining popularity for added security. The document outlines security considerations for container hosts, images, runtimes and usability, and discusses emerging projects aimed at further enhancing security such as Project Grafeas, libentitlements, and LinuxKit. It concludes that open source projects now provide tools to securely isolate containers with defense-in-depth, and vendors offer solutions to guide users in configuring container platform security.

1. @estesp
IT’S 2018.
ARE MY CONTAINERS
SECURE YET?!
DevOps Pro Vilnius

2. @estesp
HELLO!
2
Phil Estes
> Office of the CTO, IBM Cloud
> Docker Captain
> Docker engine contributor
> Containerd and Moby Project
maintainer

3. @estesp
1.
A BRIEF OVERVIEW
Easing into containers: the what and why of security
3

4. @estesp
CONTAINERS AREN’T REAL (@jessfraz)
◈ Containers on Linux are an assembly of Linux kernel
isolation features
◈ These features have been developed independently
◈ Significant use and testing has improved these areas
over the past several years, but...
4
CGROUPS + NAMESPACES + SECURITY FEATURES
Memory,
CPU,
Disk
Mount, User,
Network, UTS,
IPC, Cgroup..
AppArmor,
SELinux,
Seccomp, Caps

5. @estesp
CONTAINERS AREN’T PERFECT
◈ Kernel isolation features continue to have CVEs
◈ Hedging on lightweight virtualization has gained
popularity (e.g. Hyper.sh, Intel Clear Containers)
◈ Docker and Kubernetes have added other elements to
“containers” that require security awareness:
◆ Container Images (mini OS images!)
◆ Container Networking
◆ Storage/Volumes/Secrets
◆ Orchestration/Distributed Systems security
5

6. @estesp
WHAT DO WE WANT FROM CONTAINERS?
> Isolation from host
> Security from other tenants
> Resource limits
> Secure SW supply chain
> Immutability
> Repeatability/reproducability
6

7. @estesp
SECURITY HAS IMPROVED SIGNIFICANTLY
◈ Docker focused on security in 2016-2017
◆ Application secrets, better “sane by default” policies
(AppArmor, Seccomp), User namespaces, Swarm mode node
security, image signing, secure image format/protocol
◈ Security vendor offerings around containers has
exploded in the past 24 months
◈ Project Grafaes, Entitlements, Kubernetes security
improvements are all recent advancements
◈ Defense-in-depth model is a valuable approach
7

8. @estesp
2.
HOST SECURITY
Container runtimes run on a Linux (or Windows) server
8

9. @estesp
HOST OS SECURITY
◈ Containers use a shared kernel
◆ Escapes through kernel would
affect all containers
◆ --privileged should be used
very sparingly
◈ Root on container runtime most
likely means root on system
◈ Tenancy: single tenancy per
node? Multi-tenancy with VMs?
9
BARE METAL {or} VM
HOST OS/KERNEL
CONTAINER RUNTIME
{ orchestrator }
CTR CTR CTR
Image
Registry

10. @estesp
3.
CONTENT SECURITY
Containers images are an important part of a security
posture
10

11. @estesp
CONTAINER IMAGE/CONTENT SECURITY
◈ (Base) Images updated regularly
◆ Use image scanning tools
◆ Don’t treat containers like VMs!
◈ Least privilege mentality:
◆ No root user
◆ Minimal, read-only if possible
◆ Drop privileges
◈ Image signing & provenance
(Docker Content Trust/Notary)
11
BARE METAL {or} VM
HOST OS/KERNEL
CONTAINER RUNTIME
{ orchestrator }
CTR CTR CTR
Image
Registry

12. @estesp
4.
RUNTIME SECURITY
What security capabilities exist for container runtimes?
12

13. @estesp
CONTAINER RUNTIME/ORCHESTRATOR SECURITY
◈ Understand deployment choices
◆ RBAC? Who has access to API?
◆ Storage, Network, Volume plugins
◆ Feature switches (e.g. require
image signing)
◈ Will you use lightweight
virtualization to improve
container isolation?
◈ Secrets implementation
13
BARE METAL {or} VM
HOST OS/KERNEL
CONTAINER RUNTIME
{ orchestrator }
CTR CTR CTR
Image
Registry

14. @estesp
5.
USABLE SECURITY
If security isn’t usable, then it probably won’t be used
14

15. @estesp
SECURITY WITHOUT USABILITY = NO SECURITY
> Developers will disable
security features if they get in
the way of progress and/or
successful software deployment.
> Developers won’t learn how to
enable security features. They
are too busy. Features must be
enabled with sane defaults.
15

16. @estesp
THE SECURITY CONUNDRUM
◈ Who will decide what is “secure” for your business?
◆ Myriad of choices require expertise to make decisions
◈ Developers will turn off intrusive security
◈ DevOps implications
◆ CI/CD impact (source/upstream hygiene, image scanning,
provenance/signing, promotion policies)
◆ Security vendors offering UX to bring “manageability” to
help with choices, configuration, auditing
16

17. @estesp
STEPS ON THE RIGHT PATH
◈ Container runtimes
◆ Out of the box “sane defaults”
◆ Docker (AppArmor, Seccomp profiles built-in)
◆ Kubelet choices: “secure” or “untrusted” container models
◈ An Example: Docker Swarm Mode
◆ Built-in certificate authority deployment, full TLS
intranode encryption, certificate rotation, node secure ID
◆ Kubernetes working on similar improvements, but
componentized model of K8s makes it more challenging
17

18. @estesp
6.
SECURITY FUTURES
What’s coming next? What improvements will enable better
security for the future?
18

19. @estesp
NEW PROJECTS AND NEW IDEAS
◈ Project Grafaes (potentially Kritis)
◈ libentitlement (Moby project)
◈ LinuxKit: using image labels (for
defining capabilities/etc.)
◈ Kubernetes runtimes: using
annotations to select trust level of
runtimes
19

20. @estesp
PROJECT GRAFEAS
> Open, common metadata
server API for representing
vulnerability/incident data for
container images
> Cross-vendor commitment to
publish in Grafeas format
(Google, IBM, Twistlock,
Aqua, Redhat, etc.)
20
https://github.com/grafeas/grafeas

21. @estesp
MOBY PROJECT: LIBENTITLEMENTS
> Uses concept of Android
permission model
> Developers need to
understand broad container
permission requirements
> Library will translate that to
low-level isolation constructs
21
https://github.com/moby/libentitlement
Android App Permissions:
ACCESS_WIFI_STATE
BLUETOOTH
BLUETOOTH_ADMIN
BROADCAST_STICKY
CHANGE_NETWORK_STATE
CHANGE_WIFI_MULTICAST_STAT
E CHANGE_WIFI_STATE
DISABLE_KEYGUARD
EXPAND_STATUS_BAR
GET_PACKAGE_SIZE
INSTALL_SHORTCUT INTERNET
KILL_BACKGROUND_PROCESSES
MODIFY_AUDIO_SETTINGS

22. @estesp
ANNOTATIONS: HINTS FOR CONTAINER RUNTIMES
> A Kubernetes CRI implemention can support multiple
underlying container runtimes and use annotations to
decide which one (more secure, less secure) to use
> In this case, cri-o can use lightweight virtualization
(Intel Clear Containers) for better isolation of untrusted
code
22
io.kubernetes.cri-o.TrustedSandbox: “false”Example:

23. @estesp
LINUXKIT: COMBINING SECURITY CONCEPTS
> Uses image label to specify
runtime config (e.g. security
settings for image)
> Image signing and trust
required (on by default)
> Minimal host OS (tiny
userspace, runs containers for
services)
23
https://github.com/linuxkit/linuxkit
https://github.com/linuxkit/kubernetes

24. @estesp
7.
SUMMARY
Conclusions on the state of container security
24

25. @estesp
THE STATE OF CONTAINER SECURITY
◈ Vastly improved from early days of Docker engine
◈ Open source projects have all the right knobs for
securing and isolating containers
◈ Vendors providing turnkey solutions for those who
want a guided UX to security
◈ Work continues to make security more usable, with
sane defaults for your platform
25

26. @estesp
WHAT CAN YOU DO?
◈ Implement “defense in depth”
◆ If one layer of your protection fails, another one may either
limit blast radius or also defend against the attack
◈ Understand building blocks of container security
◆ Host
◆ Runtime
◆ Image
◆ Platform-provided (Kubernetes, networking, etc.)
26

27. @estesp
27
THANKS!
@estesp
github.com/estesp
estesp@gmail.com
https://integratedcode.us
Slack/IRC: estesp
DevOps Pro Vilnius

28. @estesp
CREDITS
Special thanks to all the people who made and
released these awesome resources for free:
◈ Presentation template by SlidesCarnival
28