Docker open source.

1. Open Source Docker
John Willis
Director of Ecosystem Development

2. About Me
• One of the founding members of the “Devops”
movement.
• Author of the “Devops Handbook”.
• Author of the “Introduction to Devops” on
Linux Foundation edX.
• Podcaster at devopscafe.org
• Devops Enterprise Summit - Cofounder
• Found of Socketplane (Acquired by Docker)
• Formally Director of Devops at Dell
• Formally Director at Chef
• 10 Startups over 25 years
2
Github: botchagalupe/my-presentations
Twitter: botchagalupe
Wechat: botchagalupe

3. Devops Practices and Patterns
• Continuous Delivery
• Everything in version control
• Small batch principle
• Trunk based deployments
• Manage flow (WIP)
• Automate everything
• Culture
• Everyone is responsible
• Done means released
• Stop the line when it breaks
• Remove silos
3
itrevolution.com/devops-handbook
http://www.bookdepository.com/

4. The world needs
tools of mass innovation

5. A programmable Internet would be the ultimate
tool of mass innovation

6. Let’s eliminate friction in the development cycle

7. A commercial product,
built on
a development platform,
built on
infrastructure,
built on
standards.
Docker is building a stack to program the Internet

8. 8
Docker Project Sponsor
•Primary authors, contributor maintainer
•6B+ Downloads, 3000+ Contributors, 500,000+ Applications
•100’s of ecosystem partners
•Millions of developers use Docker. Millions of servers run Docker
Commercial Docker Solutions
•Integrated solutions to build, ship, run Docker at scale
• Orchestration, registry, security, workflow, control plane
• CaaS (containers as a service)
•Official providers of commercial technical support
•10K’s cloud customers, 300+ F500 customers
About Docker, Inc.
Gerber, Anna. “The State of Containers and the Docker
Ecosystem: 2015” O’Reilly, September 2015
Docker users already
running in production
60%

9. China is part of Worldwide Docker Community
Meetups in Beijing, Changsha, Chengdu, Chongqing, Dalian, Fuzhou,
Guangzhou, Hangzhou, Hong Kong, Nanjing, Qingdao, Shanghai, Shenzen,
Suzhou, Tianjin, Wuhan, Xi’an, Xiamen, and Zhuhai

10. The Docker ecosystem
Dev Tools
Official Repositories
Operating Systems
Big Data
Service Discovery
Build / Continuous Integration
Configuration Management
Consulting &Training
Management
Storage
Clustering & Scheduling
Networking
Infrastructure & Service Providers
Storage
Security
Monitoring & Logging
10

11. Docker Platform

12. COMPARING CONTAINERS AND VIRTUAL MACHINES

13. Isolation using Linux kernel features
namespaces
● pid
● mnt
● net
● uts
● ipc
● user
cgroups
● memory
● cpu
● blkio
● devices

14. Image layers

15. Docker Engine extensibility and plugins
• Built in orchestration expands the opportunity for
the plugin to manage swarm wide vs a single
Engine
• Updated architecture standardizes plugin process
for ecosystem partners
– Benefits users and vendors
– Standardized process of granting plugin permissions
– Containerized plugins on roadmap
Docker Engine
Networking
Swarm Mode
Volumes
Plugins

16. Developer experience

17. 1. Get out of the way
The best tools…
2. Adapt to you
3. Make the
powerful simple

18. Docker for Mac Docker for Windows

19. Orchestration

20. Introducing the best way to
orchestrate Docker: Docker.
Docker 1.12: now with orchestration built-in.

21. Swarm mode
Service API
Node identity
Built-in routing mesh
Docker 1.12: now with orchestration built-in.

22. Using the beta? You already have 1.12 installed.
> docker swarm init
> docker service create

23. Docker CaaS

24. BUILD
Development Environments
SHIP
Registry: Secure Content &
Collaboration
RUN
Control Plane: Deploy,
Orchestrate, Manage, Scale
Networking Volumes MonitoringLoggingConfig MgtCI/CD
IT Operations
Developers IT Operations
Docker CaaS Workflow

25. Docker Containers as a Service platform
25
BUILD
Developer Workflows
SHIP
Registry Services
RUN
Management
Docker for Mac and Windows Docker Trusted Registry Docker Universal Control Plane
Docker Cloud
Docker Container Engine
Ecosystem Plugins and Integrations

26. Plumbing

27. 2013-05
2013-06
2013-07
2013-08
2013-09
2013-10
2013-11
2013-12
2014-01
1,000,000
0
2014-02
2014-03
2014-04
2014-05
2014-06
2014-07
2014-08
2014-09
2014-10
2014-11
2014-12
2015-01
2015-02
2015-03
2015-04
2015-05
2015-06
2015-07
2015-08
2015-09
2015-10
2015-11
2015-12
2016-01
1,000,000,000
~
10,000,000
9,000,000
8,000,000
7,000,000
6,000,000
5,000,000
4,000,000
3,000,000
2,000,000
6,000,000,000
5,750,000,000
5,500,000,000
5,250,000,000
5,000,000,000
4,750,000,000
4.500,000,000
4,250,000,000
4,000,000,000
3,750,000,000
3,500,000,000
3,250,000,000
3,000,000,000
2,750,000,000
2,500,000,000
2,250,000,000
2,000,000,000
1,750,000,000
1,500,000,000
1,250,000,000

28. 2013-05
2013-06
2013-07
2013-08
2013-09
2013-10
2013-11
2013-12
2014-01
1,000,000
0
2014-02
2014-03
2014-04
2014-05
2014-06
2014-07
2014-08
2014-09
2014-10
2014-11
2014-12
2015-01
2015-02
2015-03
2015-04
2015-05
2015-06
2015-07
2015-08
2015-09
2015-10
2015-11
2015-12
2016-01
~
2016-09
1,000,000,000
~
10,000,000
9,000,000
8,000,000
7,000,000
6,000,000
5,000,000
4,000,000
3,000,000
2,000,000
6,000,000,000
5,750,000,000
5,500,000,000
5,250,000,000
5,000,000,000
4,750,000,000
4.500,000,000
4,250,000,000
4,000,000,000
3,750,000,000
3,500,000,000
3,250,000,000
3,000,000,000
2,750,000,000
2,500,000,000
2,250,000,000
2,000,000,000
1,750,000,000
1,500,000,000
1,250,000,000
Notary
runC •
containerd •
HyperKit , VPNKit, DataKit •
SwarmKit •
libcontainer •
libnetwork • • Docker 1.8 : Docker Content Trust
• Docker for Mac
Docker for Windows
• Docker 1.12
with built-in
orchestration
• Docker 0.9 : Pluggable execution
• Docker 1.7 : Multi-Host Networking
• Docker 1.11:
OCI support

29. runC

30. RunC
The universal container runtime
https://runc.io

31. Open Container Initiative (OCI)
An open governance
structure for creating
open industry
standards: a common
container runtime and
image format.
•A Linux Foundation Collaborative Project
•Free from control by any particular vendor’s
specific cloud stack or ecosystem
•Includes a specification, reference
runtime* and now, a specified image format
*seeded with runc + libcontainer by Docker

32. OCI Specs & Status
> Announced June 20th,
2015
> Charter signed on
December 8th, 2015
> 49 current member
companies
> Both specifications
nearing 1.0 release
targets
https://opencontainers.org
https://github.com/opencontainers
> Runtime specification: Release 1.0.0-rc2 / September
2016
https://github.com/opencontainers/runtime-spec/releases/tag/v1.0.0-rc2
1. Very close to an official 1.0 release of the runtime spec
2. Includes required core for Linux, Windows, and Solaris
> Image format specification: Release 0.5.0 / September
2016
https://github.com/opencontainers/image-spec/releases/tag/v0.5.0
1. Seeded with Docker registry v2.2 specification
2. v1.0.0-rc1 release being voted/approved on mailing list

33. runc: An open innovation platform for containers
Implement low-level container features
Operating system level features should be defined in the OCI runtime specification
New capabilities (PID cgroup controls, checkpoint/restore, seccomp) implemented in
runC
INTEREST
OCI compliance/pluggable execution engine
Implement a OS/environment for containers via an OCI spec compliant binary
Examples: runz (Solaris zones), runv (hypervisor-based), Intel Clear Containers
Iterative container configuration test/debug
Simple variant of “Docker-like” containers with less friction for quick modifications
Low bar for dependencies: single binary + physical rootfs bundle + JSON config
INTEREST
INTEREST

34. containerd
A daemon to control runC
built for performance and density
http://containerd.tools/

35. containerd

36. Docker 1.11

37. Docker for Mac

38. Docker for Mac architecture
(simplified)

39. Hypervisor
Framework
vmnet Framework
Docker Container Engine
Hypervisor
Linux
VPN
Data
Service
Interface
Client
Libraries
Admin GUI
CLI
Security Sandbox
Docker for Mac internals

40. Unikernels
http://unikernel.org/

41. Hypervisor
Framework
vmnet Framework
Docker Container Engine
Hyperkit
Linux
VPNKit
DataKit
Client
Libraries
Admin GUI
CLI
Security Sandbox
Improving Docker with unikernel tech

42. InfraKit

43. Problem:
Managing Docker on different
infrastructure is
difficult and not portable.

44. Consistent User Experience
44
How do we handle updates to a cluster??

45. Docker for AWS
EBS ELB
Container Engine
Storage plugin
Infrastructure Management
Network
plugin
Orchestration
IAM
CloudFormation
EC2VPC
Admin
interface
Linux
User Applications / Services

46. Docker for AWS
EBS ELB
Container Engine
Storage plugin
InfraKit
Network
plugin
Orchestration
IAM
CloudFormation
EC2VPC
Admin
interface
Linux
User Applications / Services

47. InfraKit
A toolkit for building declarative, self-
healing infrastructure.

48. Declarative
• JSON configuration for desired infrastructure state:
• Specification of instances — vm image, instance type, etc.
• Group properties — size, logical identifiers, etc.
• Design patterns encourage
• encapsulation
• composition
• Config is input to all operations — system figures out what to do
48

49. Self-healing
• Composed of a set of active components / processes that
• monitor infrastructure state
• detect state divergence
• take actions
• Continuous monitoring and reconciliation — always on
• No downtime — rolling update
49

50. Toolkit
• Primitives for managing collections of resources
• create, scale, destroy
• rolling update
• Abstractions & Developer SPI
• Group - manages collection of resources
• Instance - describes the physical resource
• Flavor - extra semantics for handling instances
• A collection of executable, active components — plugins
• Initially, Go daemons in the toolkit
• Soon, easy management via Docker Plugins (runc)

51. Architecture

52. Instance Plugin
• Spec: specification / model of an instance (e.g. vagrant, EC2):
• Logical ID, Init, Tags, and attachment
• Platform-specific properties
• Methods:
• /Instance.Validate
• /Instance.Provision
• /Instance.Destroy
• /Instance.DescribeInstances
• Examples: instance plugins for EC2, Azure VM, Vagrant, …
52

53. Flavor Plugin
• Gives more context about the group members:
• Size, or list of Logical ID’s (e.g. IP addresses for ‘pets’)
• Application-specific notions of ‘health’
Is the node not only present but also joined a swarm?
• Methods:
• /Flavor.Validate
• /Flavor.Prepare
• /Flavor.Healthy
• Examples: flavor for Zookeeper members, Docker swarm nodes
53

54. Group Plugin
• Main entry point for user interaction:
• Create, describe update, update, destroy
• Config JSON is always the input
• Composed of Instance and Flavor — mix and match to
manage cattle (fungible) or pets (special)
• Methods:
• /Group.Watch
• /Group.Unwatch
• /Group.Inspect
54
• /Group.DescribeUpdate
• /Group.Update
• /Group.StopUpdate
• /Group.Destroy

55. Configuration
Example config file (zk.conf): Group configuration = Instance + Flavor
{
"Properties": {
/* raw configuration
*/
}
}
{
"groups" : {
"my_zookeeper_nodes" : {
"Properties" : {
"Instance" : {
"Plugin": "instance-vagrant",
"Properties": {
"Box": "bento/ubuntu-16.04"
}
},
"Flavor" : {
"Plugin": "flavor-zookeeper",
"Properties": {
"type": "member",
"IPs": ["192.168.1.200", "192.168.1.201",
"192.168.1.202"]
}
}
}
}
}

56. Today
56
• InfraKit is just getting started… only
primitives for working with groups like
clusters of hosts
• But we have big plans
• Improve group management strategies
• More resource types — networking, load
balancers, storage…
• A cohesive framework for active
management of infrastructure — physical,
virtual, or containers

57. Get Involved
• Help define and implement new and interesting plugins
• Instance plugins for different infrastructure providers
• Flavor plugins for systems like etcd or mysql clusters
• Group controller plugins — metrics-driven auto scaling
and more
• Help define interfaces and implement new infrastructure
resource types — load balancers, networks and storage
volume provisioners
57

58. More Info
• Github:
https://github.com/docker/infrakit
• A quick tutorial:
https://github.com/docker/infrakit/blob/master/docs/
tutorial.md
58

59. Docker Open Source Projects
Summary

63. Extra Slides

64. Docker on Windows