Frank Macreery, Aptible CTO, gives Ruby devs an introduction to Docker, simplifying service-oriented architecture, wrapping databases in a uniform API, and achieving the Holy Grail of dev/prod parity.

1. Docker for Ruby
Developers
NYC.rb, August 2015
Frank Macreery
CTO, Aptible
@fancyremarker
https://speakerdeck.com/fancyremarker

3. Docker
What’s it all about?

6. Docker
Containers: The new virtual machines?

7. https://www.docker.com/whatisdocker
Then: Virtual Machines

8. https://www.docker.com/whatisdocker

9. https://www.docker.com/whatisdocker
Now: Containers

10. https://www.docker.com/whatisdocker

11. Getting Started with
Docker

12. boot2docker

13. Kitematic

14. Getting Started with
Docker
Install boot2docker via
http://boot2docker.io/#installation
echo 'eval "$(boot2docker shellinit)"' >> ~/.bashrc

15. Getting Started with
Docker
Alternatively (if you have Homebrew and
VirtualBox installed)…
brew install boot2docker
brew install docker

16. Getting Started with
Docker
Images and containers

17. docker pull quay.io/aptible/nginx:latest

18. docker pull quay.io/aptible/nginx:latest
Reference to a Docker image

19. docker pull quay.io/aptible/nginx:latest
Image "repositories" can have many
"tags"

20. docker pull quay.io/aptible/nginx:latest
Pulls an image from a Docker
"registry"

21. A single
Docker image
consists of
many "layers"

22. docker images

23. docker run -d -p 80:80 -p 443:443
quay.io/aptible/nginx:latest

24. docker run -d -p 80:80 -p 443:443
quay.io/aptible/nginx:latest
Launches a new container from an
existing image

25. docker run -d -p 80:80 -p 443:443
quay.io/aptible/nginx:latest
Forward container ports to the
host (host port:container port)

26. docker run -d -p 80:80 -p 443:443
quay.io/aptible/nginx:latest
Run container in background

27. docker ps

30. Simplifying SOA
Service-oriented architectures without the
complexity

31. Dev/prod parity?

33. What makes dev/prod
parity so hard?

34. What makes dev/prod
parity so hard?
1 production deployment, many development/
staging environments

35. What makes dev/prod
parity so hard?
SOA simplifies each service’s responsibilities,
but often at the cost of additional deployment
complexity

36. What makes dev/prod
parity so hard?
The more services you have, the harder it is to
achieve dev/prod parity

37. What makes dev/prod
parity so hard?
The more engineers you have, the harder it is to
standardize dev parity

41. README != Automation

42. Dev/prod parity via Docker

43. Dev/prod parity via Docker
Define services in terms of Docker images

44. # docker-compose.yml for Aptible Auth/API
auth:
build: auth.aptible.com/
ports: "4000:4000"
links:
- postgresql
environment:
RAILS_ENV: development
DATABASE_URL: postgresql://postgresql/aptible_auth_development
api:
build: api.aptible.com/
ports: "4001:4001"
links:
- redis
- postgresql
environment:
RAILS_ENV: development
DATABASE_URL: postgresql://postgresql/aptible_api_development
REDIS_URL: redis://redis
APTIBLE_AUTH_ROOT_URL: http://docker:4000
redis:
image: quay.io/aptible/redis:latest
ports: "6379:6379"
postgresql:
image: quay.io/aptible/postgresql:aptible-seeds
ports: "5432:5432"

45. # docker-compose.yml for Aptible Auth/API
auth:
build: auth.aptible.com/
ports: "4000:4000"
links:
- postgresql
environment:
RAILS_ENV: development
DATABASE_URL: postgresql://postgresql/aptible_auth_development
api:
build: api.aptible.com/
ports: "4001:4001"
links:
- redis
- postgresql
environment:
RAILS_ENV: development
DATABASE_URL: postgresql://postgresql/aptible_api_development
REDIS_URL: redis://redis
APTIBLE_AUTH_ROOT_URL: http://docker:4000
redis:
image: quay.io/aptible/redis:latest
ports: "6379:6379"
postgresql:
image: quay.io/aptible/postgresql:aptible-seeds
ports: "5432:5432"

46. # docker-compose.yml for Aptible Auth/API
auth:
build: auth.aptible.com/
ports: "4000:4000"
links:
- postgresql
environment:
RAILS_ENV: development
DATABASE_URL: postgresql://postgresql/aptible_auth_development
api:
build: api.aptible.com/
ports: "4001:4001"
links:
- redis
- postgresql
environment:
RAILS_ENV: development
DATABASE_URL: postgresql://postgresql/aptible_api_development
REDIS_URL: redis://redis
APTIBLE_AUTH_ROOT_URL: http://docker:4000
redis:
image: quay.io/aptible/redis:latest
ports: "6379:6379"
postgresql:
image: quay.io/aptible/postgresql:aptible-seeds
ports: "5432:5432"

47. # docker-compose.yml for Aptible Auth/API
auth:
build: auth.aptible.com/
ports: "4000:4000"
links:
- postgresql
environment:
RAILS_ENV: development
DATABASE_URL: postgresql://postgresql/aptible_auth_development
api:
build: api.aptible.com/
ports: "4001:4001"
links:
- redis
- postgresql
environment:
RAILS_ENV: development
DATABASE_URL: postgresql://postgresql/aptible_api_development
REDIS_URL: redis://redis
APTIBLE_AUTH_ROOT_URL: http://docker:4000
redis:
image: quay.io/aptible/redis:latest
ports: "6379:6379"
postgresql:
image: quay.io/aptible/postgresql:aptible-seeds
ports: "5432:5432"

48. Dev/prod parity via Docker
Use the same service/image configuration in
production as in development (Docker
Compose, Swarm, Kubernetes…)

49. Containerized SSL
Infrastructure management made easy

50. Elastic Load Balancer (ELB)
EC2 Instance EC2 Instance
NGiNX NGiNX

51. TCP/HTTPSTCP/HTTPS
HTTP HTTP

52. How to configure NGiNX with
multiple dynamic upstreams?
Chef?
Salt?
Ansible?

53. ENV configuration
$UPSTREAM_SERVERS

54. docker run -d -p 80:80 -p 443:443
-e UPSTREAM_SERVERS=docker:4000,docker:4001
quay.io/aptible/nginx:latest

55. docker run -d -p 80:80 -p 443:443
-e UPSTREAM_SERVERS=docker:4000,docker:4001
quay.io/aptible/nginx:latest

56. ENV configuration
Makes testing easier

57. https://github.com/sstephenson/bats

58. # Dockerfile
# Install and configure NGiNX...
# ...
ADD test /tmp/test
RUN bats /tmp/test
https://github.com/aptible/docker-nginx
Image: quay.io/aptible/nginx

59. #!/usr/bin/env bats
# /tmp/test/nginx.bats
@test "It should accept a list of UPSTREAM_SERVERS" {
simulate_upstream
UPSTREAM_SERVERS=localhost:4000 wait_for_nginx
run curl localhost 2>/dev/null
[[ "$output" =~ "Hello World!" ]]
}

60. #!/usr/bin/env bats
# /tmp/test/nginx.bats
@test "It should accept a list of UPSTREAM_SERVERS" {
simulate_upstream
UPSTREAM_SERVERS=localhost:4000 wait_for_nginx
run curl localhost 2>/dev/null
[[ "$output" =~ "Hello World!" ]]
}

61. @test "It should accept a list of UPSTREAM_SERVERS" {
simulate_upstream
UPSTREAM_SERVERS=localhost:4000 wait_for_nginx
run curl localhost 2>/dev/null
[[ "$output" =~ "Hello World!" ]]
}
simulate_upstream() {
nc -l -p 4000 127.0.0.1 < upstream-response.txt
}

62. ENV configuration
Abstracts implementation details:
could be NGiNX, HAProxy, …

63. ENV configuration
Simplifies configuration management: central
store doesn’t need to know parameters in advance

64. ENV configuration
$UPSTREAM_SERVERS
$FORCE_SSL
$DISABLE_WEAK_CIPHER_SUITES
(…)

65. Vulnerability Response
Fix, test, docker push, restart

66. Heartbleed

67. Heartbleed
POODLEbleed

68. Heartbleed
POODLEbleed
xBleed???

69. Integration Tests
Document and test every vulnerability response

70. #!/usr/bin/env bats
# /tmp/test/nginx.bats
@test "It should pass an external Heartbleed test" {
install_heartbleed
wait_for_nginx
Heartbleed localhost:443
uninstall_heartbleed
}

71. @test "It should pass an external Heartbleed test" {
install_heartbleed
wait_for_nginx
Heartbleed localhost:443
uninstall_heartbleed
}
install_heartbleed() {
export GOPATH=/tmp/gocode
export PATH=${PATH}:/usr/local/go/bin:${GOPATH}/bin
go get github.com/FiloSottile/Heartbleed
go install github.com/FiloSottile/Heartbleed
}

72. Integration tests happen
during each image build

73. Integration tests happen
during each image build
Images are built automatically via Quay
Build Triggers

75. Integration tests happen
during each image build
Build status is easy to verify at a glance

77. Integration tests happen
during each image build
Quay Time Machine lets us roll back an
image to any previous state

79. Database Deployment
Standardizing an "API" across databases

81. New databases mean
new dependencies

82. New databases mean
new dependencies
How to document setup steps for engineers?

83. New databases mean
new dependencies
How to deploy in production?

84. New databases mean
new dependencies
How to perform common admin tasks?
Backups? Replication? CLI access? Read-only
mode?

85. Wrap Databases in a
Uniform API
Standardizing an "API" across databases

86. #!/bin/bash
# run-database.sh
command="/usr/lib/postgresql/$PG_VERSION/bin/postgres -D "$DATA_DIRECTORY" -c config_file=/etc/postgresql/
$PG_VERSION/main/postgresql.conf"
if [[ "$1" == "--initialize" ]]; then
chown -R postgres:postgres "$DATA_DIRECTORY"
su postgres <<COMMANDS
/usr/lib/postgresql/$PG_VERSION/bin/initdb -D "$DATA_DIRECTORY"
/etc/init.d/postgresql start
psql --command "CREATE USER ${USERNAME:-aptible} WITH SUPERUSER PASSWORD '$PASSPHRASE'"
psql --command "CREATE DATABASE ${DATABASE:-db}"
/etc/init.d/postgresql stop
COMMANDS
elif [[ "$1" == "--client" ]]; then
[ -z "$2" ] && echo "docker run -it aptible/postgresql --client postgresql://..." && exit
psql "$2"
elif [[ "$1" == "--dump" ]]; then
[ -z "$2" ] && echo "docker run aptible/postgresql --dump postgresql://... > dump.psql" && exit
pg_dump "$2"
elif [[ "$1" == "--restore" ]]; then
[ -z "$2" ] && echo "docker run -i aptible/postgresql --restore postgresql://... < dump.psql" && exit
psql "$2"

87. #!/bin/bash
# run-database.sh
command="/usr/lib/postgresql/$PG_VERSION/bin/postgres -D "$DATA_DIRECTORY" -c config_file=/etc/postgresql/
$PG_VERSION/main/postgresql.conf"
if [[ "$1" == "--initialize" ]]; then
chown -R postgres:postgres "$DATA_DIRECTORY"
su postgres <<COMMANDS
/usr/lib/postgresql/$PG_VERSION/bin/initdb -D "$DATA_DIRECTORY"
/etc/init.d/postgresql start
psql --command "CREATE USER ${USERNAME:-aptible} WITH SUPERUSER PASSWORD '$PASSPHRASE'"
psql --command "CREATE DATABASE ${DATABASE:-db}"
/etc/init.d/postgresql stop
COMMANDS
elif [[ "$1" == "--client" ]]; then
[ -z "$2" ] && echo "docker run -it aptible/postgresql --client postgresql://..." && exit
psql "$2"
elif [[ "$1" == "--dump" ]]; then
[ -z "$2" ] && echo "docker run aptible/postgresql --dump postgresql://... > dump.psql" && exit
pg_dump "$2"
elif [[ "$1" == "--restore" ]]; then
[ -z "$2" ] && echo "docker run -i aptible/postgresql --restore postgresql://... < dump.psql" && exit
psql "$2"

88. --initialize: Initialize data directory
--client: Start a CLI client
--dump: Dump database to STDOUT
--restore: Restore from dump
--readonly: Start database in RO mode

89. db-launch ()
{
container=$(head -c 32 /dev/urandom | md5);
passphrase=${PASSPHRASE:-foobar};
image="${@: -1}";
docker create --name $container $image
docker run --volumes-from $container
-e USERNAME=aptible -e PASSPHRASE=$passphrase
-e DB=db $image --initialize
docker run --volumes-from $container $@
}
http://bit.ly/aptible-dblaunch

90. docker create --name $container $image
http://bit.ly/aptible-dblaunch
1. Create "volume container"

91. docker run --volumes-from $container
-e USERNAME=aptible -e PASSPHRASE=$passphrase
-e DB=db $image --initialize
http://bit.ly/aptible-dblaunch
2. Initialize database data volume

92. docker run --volumes-from $container $@
http://bit.ly/aptible-dblaunch
3. Run database

94. Thank you

95. @fancyremarker
frank@aptible.com
https://speakerdeck.com/fancyremarker