1.
@mfdii
Michael Ducy, Sysdig, @mfdii
Securing your
Container Environment
with Open Source

2.
@mfdii
Layers Container Security
Infra, Build, Runtime
Container Security Challenges
Open Source Tools For:
- Infra
- Build
- Runtime
Container Security Architecture
Agenda

3.
@mfdii
Layers of Container Security
Runtime
Build
Infrastructure

4.
@mfdii
Infrastructure
Host Security
Networking
Cluster Security
Container Runtime

5.
@mfdii
Build
Image/Software Provenance
- Signed Images/Layers
- Artifact Signing
Vulnerability Management
- Upstream OS
- Application Vulnerabilities

6.
@mfdii
Runtime
Service/Container Admittance
Secure Secrets
Anomaly Detection
Forensics

7.
@mfdii
Decisions Pushed to Edge
Ephemeral Nature of Containers
Attack Surface
Resource Isolation
Challenges of Container Security

8.
@mfdii
Infrastructure Security
Network Storage
Host
Cluster
Container Runtime

9.
@mfdii
Infrastructure Security
Cluster:
- RBAC, Security Policies, Affinity
Host/Container Runtime:
- Seccomp, SELinux, AppArmor, Resource Constraints
Network:
- Service Mesh, Network Policy, Network Filtering

10.
@mfdii
Security Policies
Security Policies define:
- Access to host resources:
- Filesystem, Host Network, Namespaces
- User/Group of Container
- Read Only Filesystem
- Linux capabilities available:
- http://man7.org/linux/man-pages/man7/capabilities.7.html
- Seccomp, AppArmor, or SELinux profiles

11.
@mfdii
Linux Security Modules
SELinux
System wide execution policy
Apparmor
System wide execution policy, focused on processes
Seccomp
Per process system call isolation

12.
@mfdii
LSMs
$ docker run --security-opt "apparmor=<profile>"
$ docker run --security-opt
seccomp=/path/to/seccomp/profile.json

13.
@mfdii
Security Policies
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: restricted
annotations:
seccomp.security.alpha.kubernetes.io/allowedProfileNames: 'docker/default'
apparmor.security.beta.kubernetes.io/allowedProfileNames: 'runtime/default'
seccomp.security.alpha.kubernetes.io/defaultProfileName: 'docker/default'
apparmor.security.beta.kubernetes.io/defaultProfileName: 'runtime/default'
spec:
privileged: false
# Required to prevent escalations to root.
allowPrivilegeEscalation: false
# This is redundant with non-root + disallow privilege escalation,
# but we can provide it for defense in depth.
requiredDropCapabilities:
- ALL
# Allow core volume types.
volumes:
- 'configMap'
- 'emptyDir'
- 'projected'
- 'secret'
- 'downwardAPI'
# Assume that persistentVolumes set up by the cluster admin are safe to use.
- 'persistentVolumeClaim'
hostNetwork: false

14.
@mfdii
Container Affinity/Constraints
Affinity ensures:
- Containers with sensitive data or data processing routines are
next scheduled next to other containers
Strong labeling schema encouraged/required.

15.
@mfdii
Kubernetes Pod Affinity
apiVersion: v1
kind: Pod
metadata:
name: with-node-affinity
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/e2e-az-name
operator: In
values:
- e2e-az1
- e2e-az2
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
preference:
matchExpressions:
- key: another-node-label-key
operator: In
values:
- another-node-label-value

16.
@mfdii
Host/Container Runtime Security
CIS Benchmarks for Docker Hosts
- https://www.cisecurity.org/benchmark/docker/
- Chef’s Inspec to scan for policy violations
- https://inspec.io
- https://github.com/dev-sec/cis-docker-benchmark
- Docker Bench for Security
- https://github.com/docker/docker-bench-security

17.
@mfdii
Networking
Standard Firewall Rules/Security Groups
- Common exploit point are dashboards or API ports open.
Kubernetes:
- Network Policies, Container Networking Interface
Network Filtering
- Kernel level L3/L4/L7
- Cilium - https://cilium.io/

18.
@mfdii
Cilium
apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
description: "L3-L4 policy to restrict deathstar access to empire ships only"
metadata:
name: "rule1"
spec:
endpointSelector:
matchLabels:
org: empire
class: deathstar
ingress:
- fromEndpoints:
- matchLabels:
org: empire
toPorts:
- ports:
- port: "80"
protocol: TCP

19.
@mfdii
Cilium

20.
@mfdii
Cilium
apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
description: "L7 policy to restrict access to specific HTTP call"
metadata:
name: "rule1"
spec:
endpointSelector:
matchLabels:
org: empire
class: deathstar
ingress:
- fromEndpoints:
- matchLabels:
org: empire
toPorts:
- ports:
- port: "80"
protocol: TCP
rules:
http:
- method: "POST"
path: "/v1/request-landing"

21.
@mfdii
Cilium

22.
@mfdii
Build Security
Network Storage
Host
Cluster
App Code
App Runtime
Libraries
OS
App Code
App Runtime
Libraries
OS
App Code
App Runtime
Libraries
OS
Container Runtime

23.
@mfdii
Build Security
Image/Artifact Provenance:
- Artifact signing, trusted registries, admittance control
Vulnerability Management:
- Image scanning, OS libraries, application libraries

24.
@mfdii
Image Scanning
Clair
- Static analysis of vulnerabilities in application containers.
- Focuses on Operating System packages and libraries
- https://github.com/coreos/clair
Anchore
- Analyzes container images against user defined policies.
- https://github.com/anchore

25.
@mfdii
CoPilot & Openhub
CoPilot
- Open Source Application Dependency Vulnerability Management
- https://copilot.blackducksoftware.com/
- Incorporate into your build process
OpenHub
- Compare open source project usage, and project health
- https://www.openhub.net/

26.
@mfdii
Notary & Portieris
Notary
- Signs collections of digital content (Artifacts)
- Project from Docker - Docker Content Trust
- Implementation of The Update Framework
- https://github.com/theupdateframework/
Portieries
- Kubernetes Admission controller for enforcing Content Trust
- https://github.com/IBM/portieris
- https://schd.ws/hosted_files/kccnceu18/41/kubernetes-notary-tuf.pdf
- https://www.youtube.com/watch?v=JK70k_B87mw

27.
@mfdii
Notary & Portieris

28.
@mfdii
Runtime Security
Network Storage
Host
Cluster
App Code
App Runtime
Libraries
OS
App Code
App Runtime
Libraries
OS
App Code
App Runtime
Libraries
OS
Container Runtime

29.
@mfdii
Runtime Security
Service/Container Admittance
- What’s Allowed to Run/Join a Service
Secure Secrets
- How do applications authenticate
Anomaly Detection
- Is my runtime environment being tampered with?
Forensics
- What happened if something was compromised?

30.
@mfdii
Service Identity
How can you verify a service is who it says it is?
SPIFFE
- Secure Production Identity Framework For Everyone
- Cryptographically verifiable Service IDs
- https://github.com/spiffe/spiffe

31.
@mfdii
Service Identity

32.
@mfdii
Anomaly Detection
- Containers are isolated processes.
- Processes are “scoped” as to what’s expected.
- Container images are immutable, runtime environments
often aren’t.
- How do you detect “abnormal” behavior.

33.
@mfdii
Sysdig Falco
A behavioral activity monitor
•Detects suspicious activity defined by a set of rules
•Uses Sysdig’s flexible and powerful filtering expressions
With full support for containers/orchestration
•Utilizes sysdig’s container & orchestrator support
And flexible notification methods
•Alert to files, standard output, syslog, programs
Open Source
•Anyone can contribute rules or improvements

34.
@mfdii
Quick examples
A shell is run in a container container.id != host and proc.name = bash
Overwrite system binaries
fd.directory in (/bin, /sbin, /usr/bin, /usr/sbin)
and write
Container namespace change
evt.type = setns and not proc.name in
(docker, sysdig)
Non-device files written in /dev
(evt.type = create or evt.arg.flags contains O_CREAT)
and proc.name != blkid and fd.directory = /dev and
fd.name != /dev/null
Process tries to access camera
evt.type = open and fd.name = /dev/video0
and not proc.name in (skype, webex)

35.
@mfdii
Falco architecture
falco_probe
Kernel
Module
Kernel
User
Syscalls
Sysdig Libraries
`
Events
Alerting
Falco Rules
Suspicious
Events File
Syslog
Stdout
Filter Expression
Shell

36.
@mfdii
Falco Rules
25 common rules available OOTB
Focused on common container best practices:
■ Writing files in bin or etc directories
■ Reading sensitive files
■ Binaries being executed other than CMD/ENTRYPOINT

37.
@mfdii
Falco rules
.yaml file containing Macros, Lists, and Rules
- macro: bin_dir
condition: fd.directory in (/bin, /sbin, /usr/bin, /usr/sbin)
- list: shell_binaries
items: [bash, csh, ksh, sh, tcsh, zsh, dash]
- rule: write_binary_dir
desc: an attempt to write to any file below a set of binary directories
condition: bin_dir and evt.dir = < and open_write and not package_mgmt_procs
output: "File below a known binary directory opened for writing
(user=%user.name command=%proc.cmdline file=%fd.name)"
priority: WARNING

38.
@mfdii
Active Security with Falco, NATS,
and Kubeless
Falco NATS Kubeless
Detects abnormal event,
Publishes alert to NATS
Subscribers receive
Falco Alert through
NATS Server
Kubeless receives
Falco Alert, firing a
function to delete the
offending Kubernetes
Pod

39.
@mfdii
Join the community
• Website
•http://www.sysdig.org/falco
• Public Slack
•http://slack.sysdig.com/
•https://sysdig.slack.com/messages/falco
• Blog
•https://sysdig.com/blog/tag/falco/
• Sysdig Secure
•http://sysdig.com/product/secure

40.
@mfdii
Learn more
Github
• https://github.com/draios/falco
• Pull Requests welcome!
Wiki
• https://github.com/draios/falco/wiki
Docker Hub
• https://hub.docker.com/r/sysdig/falco/

41.
@mfdii
Thank You.
Questions?
michael@sysdig.com, @mfdii