John will fully explain what Docker is, why it is useful, how it works and most importantly discuss the pros and cons of it from real world production experience. We will then build out (from scratch) a clustered Docker environment using Rancher (machine setup done with Ansible) across multiple virtual machines in the cloud using Terraform learning tips, tricks and what is happening under the covers along the way.
Expect to leave the talk feeling confident about if Docker might make sense for your next project (or might not!) and how to get started easily if it looks like it is the right tool for the job for you.
2. About Me
Free range, sometimes organic
Full-stack Independent Consultant
@ Veritas in Roseville
Backend
DevOps (Docker, Ansible, Linux etc)
NoSql (ElasticSearch, MongoDB)
Distributed systems (RabbitMQ, Kafka etc.)
Node.js
Groovy/Spring/Java
C#
Front End
Angular.js, React.js, Knockout.js, Durandal.js, jQuery, CSS/SASS etc.
SPA development
Open Source “Street Cred”
AngularAgility
jQuery File Download
FluentKnockoutHelpers
3. Overview
Docker
How does it work
Why would I use it
Rancher
What does it give me
Building a Clustered Docker + Rancher environment from
scratch
Terraform (DigitalOcean)
Ansible
Node.js Microservice
Objective:
To leave feeling confident about if Docker might make
sense for your next project (or might not!) and how to
get started easily if it looks like it is the right tool for the
job for you.
5. Images
Internal Docker Registry
hostname: MY_REG:5000
myapp:1.0
myapp:1.1
yourapp:1.0
yourapp:1.1
…
Public Docker Registry
AKA: hub.docker.com
elasticsearch:5.0.0
elasticsearch:5.0.1
rabbitmq:3.6.4
rabbitmq:3.6.5
…
Any machine running Docker
MY_REG:5000/myapp:1.1
elasticsearch:5.0.1
may equal
when :tag not specified defaults to
6. Confused? Container vs Image
A container is an “instance” of an “immutable” image
Could be running or stopped
My machine running Docker for Mac
Loaded Images
mongo:latest
Running Containers
Image Name
mongo:latest
Container Name
myfirstmongo
Image Name
mongo:latest
Container Name
mysecondmongo
…
7. Moderate Mongo Mess
mongo:latest isn’t terribly useful to know what the version really is
There is no external/port level access to the containers
There are no volume mounts for persistent data (very bad for perf
on with high I/O applications)
If the container dies it’s not coming up again without me restarting
it
Fortunately? there is:
docker run --name=myfirstmongo --detach --publish="27017:27017" --
restart=always --volume="/some/local/path:/data/db" mongo
8. A better way: docker-compose
Tearse & readily source controlled YAML definition
docker-compose.yml
Idempotence
(to an extent)
9. docker-compose for CI/CD!
Run isolated integration testing CI/CD of your whole app
stack from anywhere! (local, Jenkins etc.)
Builds a local Dockerfile
Define DNS aliases of references
only available from my_app
stdout/err comes out to
pass/fail Jenkins build
Test command: stdout/err
comes out of container to
pass/fail the build
Mongo only addressable to
my_app at DNS “mongodb”
No stdout/err
Real live chrome/selenium
server in a container using xvfb
10. Benefits of Images & Containers
Better Isolation & Consistency with Images
Docker Repository vs. Artifactory, NPM, Nuget etc.
Debug a production image on my local machine
EX: Run 10 different YOUR_FAV_LANG apps using 10
different versions the runtime all on port 8080 on same
box*
*with a SDN (software defined network)
Security*
*When you don’t run as root, use SELinux,
sandbox volumes among other things
+Docker
11. Building images with layers
Done with a Dockerfile, lets do it!
See layers with “docker inspect IMAGE_NAME”
What we did:
image layer: alpine:latest
image layer: first_file added
image layer: second_file added
container: second-container
container: first-container
12. Layer re-creation/sharing
Docker will re-use existing layers when it can:
When a layer changes subsequent layers are invalidated otherwise they are
re-used
This effects:
Proportion of
Image Size
Changes every build
(probably)
npm install only runs
if package.json
(a dependency/package manifest)
changes!
pull/push HTTP traffic
server filesystem usage repository storage space
BUILD TIMES!
13. Docker Observations
Set up development environment quickly
with a docker-compose for a project
E2E Integration testing easily with a docker-compose
Image consistency to production
stdout/stderr aggregation
QA servers
myapp:1.2.3
PROD servers
myapp:1.2.3
DEV servers
myapp:1.2.3
- Commit
- Build
- Test
deploy
server-a
server-b
server-c
server-d
ElasticSearch
+
Kibana
stdout/err from
all containers
14. Well that was cool for DEV but…
How do I run containers on multiple machines and
orchestrate them?
How do I ensure HA (high availability)
How do I load balance HTTP/S applications
How do I schedule based on load
Does Docker actually make sense to
run real applications in PROD?
*well I have at least with less work and less
downtime than other approaches I’ve
encountered… so far
15. Partial lay of the land*
*as I see it: grain of salt please
+
16. What is ?
A really slick UI that illustrates what is going on in a
very clear manner
Actually helps you learn real Docker (full API surface
almost!) visually and then helps you script things after you
have “pointed and clicked your way to success”
Easily runs in Docker container(s)
Container orchestration/clustering support for a variety
of different platforms:
17. What is Cattle?
A relatively simple container orchestration framework that is
natively supported by Rancher
Pros
Built in layer 5 (haproxy based) load balancer that supports
scaling, rolling upgrades, rollback changes etc.
Slick SDN (Software Defined Network) does DNS based round-
robin inter-container network resolution
Simpler & quicker to get going than anything else
“3AM Googleability” is very high / vibrant community
Works with Docker rather than against it
Realistically free!
I’ve battle tested it and has worked well so far
Cons
Scheduler is rather simple / no automatic container creation
support
18. + +
Setup entire stack from scratch in a repeatable
(idempotent), clear & source controllable manner
*Some of the Rancher stuff we will “point and click our
way to success” for brevity and to show you the UI but
I’ve done it before with 100% Ansible + docker/rancher-
compose files.
Requirements
POSIX shell
DigitalOcean account with API key env variable
SSH
~/.ssh/id_rsa + ~/.ssh/id_rsa.pub setup
Ansible (get with Python, PIP)
Terraform
Web browser
19. The
Goal
docker0 docker2docker1 docker3
rancher/server
rancher/agent
rancher/agent rancher/agent rancher/agent
Idempotent Cloud VM
creation tool
Cloud VM Provider
Ubuntu
16.04
Cloud VMs
w/
Containers
Idempotent
Server
Provisioning
Tool
johnculviner/
nodejs-echo-hostname
johnculviner/
nodejs-echo-hostname
johnculviner/
nodejs-echo-hostname
johnculviner/
nodejs-echo-hostname
johnculviner/
nodejs-echo-hostname
johnculviner/
nodejs-echo-hostname
…… …
rancher haproxy
load balancer
HTTP
Traffic
+ few SSH commands
The code
https://github.com/johnculviner/doc
ker-rancher-presentation