Shipping node.js application with docker.

1. dockerize node.js application
shipping node.js application with docker
김석준 @SOCAR

2. YOU CAN DOCKERIZE
YOUR APPLICATION TOMORROW

3. WHY WHAT HOW

4. WHY WHAT HOW

5. WHY?
EASIER ENVIRONMENT SETUP
SINGLE-THREADED in MULTI-CORE
FAST AND EASY TO ROLLBACK
MICROSERVICE
VIRTUALIZATION

6. EASIER ENVIRONMENT SETUP
Application Server

7. EASIER ENVIRONMENT SETUP
TRADITIONAL WAY
$ curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
$ sudo apt-get update
$ sudo apt-get install nodejs
…
$ echo “deb http://nginx.org/packages/debian/ squeeze nginx” >> /etc/apt/sources.list
$ echo “deb-src http://nginx.org/packages/debian/ squeeze nginx” >> /etc/apt/sources.list
$ sudo apt-get install nginx
…
$ sudo apt-get install ntp vim fail2ban
…
$ vi /etc/nginx/site-enabled/node.conf
$ nginx -s reload
…
$ mkdir /home/alma/app && cd /home/alma/app
$ git pull … && npm install
$ npm run build
$ npm run serve

8. EASIER ENVIRONMENT SETUP
Application
Servers

9. EASIER ENVIRONMENT SETUP
$ curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
$ sudo apt-get update
$ sudo apt-get install nodejs
…
$ echo “deb http://nginx.org/packages/debian/ squeeze nginx” >> /etc/apt/sources.list
$ echo “deb-src http://nginx.org/packages/debian/ squeeze nginx” >> /etc/apt/sources.list
$ sudo apt-get install nginx
…
$ sudo apt-get install ntp vim fail2ban
…
$ vi /etc/nginx/site-enabled/node.conf
$ nginx -s reload
…
$ mkdir /home/alma/app && cd /home/alma/app
$ git pull … && npm install
$ npm run build
$ npm run serve
X3
TRADITIONAL WAY

10. EASIER ENVIRONMENT SETUP
Application
Many Servers

11. EASIER ENVIRONMENT SETUP
“이러려고 개발자 하려고 했나 자괴감 들고 괴로워”
TRADITIONAL WAY
“I really didn’t want to be a developer to do this”

12. SINGLE-THREADED in MULTI-CORE
MULTICORESingle-threaded
in

13. SINGLE-THREADED in MULTI-CORE
CLUSTER
var cluster = require('cluster');
if (cluster.isMaster) {
var numWorkers = require('os').cpus().length;
console.log('Master cluster setting up ' + numWorkers + ' workers...');
for (var i = 0; i < numWorkers; i++) {
cluster.fork();
}
cluster.on('online', function(worker) {
console.log('Worker ' + worker.process.pid + ' is online');
});
cluster.on('exit', function(worker, code, signal) {
console.log('Worker ' + worker.process.pid + ' died with code: ' + code + ', and signal: ' + signal);
console.log('Starting a new worker');
cluster.fork();
});
} else {
var app = require('express')();
app.all('/*', function(req, res) {
res.send('process ' + process.pid + ' says hello!').end();
})
var server = app.listen(8000, function() {
console.log('Process ' + process.pid + ' is listening to all incoming requests');
});
}
@https://www.sitepoint.com/how-to-create-a-node-js-cluster-for-speeding-up-your-apps/

14. SIMPLICITY

15. SINGLE-THREADED in MULTI-CORE
CLUSTER
TOO COMPLICATED

16. VIRTUALIZATION
STAGINGDEV
LIVE
PHYSICAL
LIVE
DB

17. VIRTUALIZATION
STAGINGLOCAL
LIVE
CLOUD
DEV
LIVE
PHYSICAL
LIVE
DB

18. VIRTUALIZATION
STAGINGLOCAL
LIVE
CLOUD
DEV
LIVE
PHYSICAL
STAGING
DB
LIVE
DB
REPLICA
DB

19. VIRTUALIZATION
STAGINGLOCAL
LIVE
CLOUD
DEV
LIVE
PHYSICAL
STAGING
DB
REPLICA
DB
LIVE
DB
environment variables
environment variables
environment variables

20. VIRTUALIZATION
STAGINGLOCAL
LIVE
CLOUD
DEV
LIVE
PHYSICAL
STAGING
DB
REPLICA
DB
environment variables
LIVE
DB #1
LIVE
DB #2
LIVE
CLOUD
LIVE
CLOUD
CRON
environment variables
environment variables

21. VIRTUALIZATION
UNABLE TO TRACK ENVIRONMENTS

22. MICROSERVICE
SOCAR
APP
API
BACK
OFFICE
API
ZEROCAR
APP
API
BACK
OFFICE
API
MAKE IT DETACHABLE

23. MICROSERVICE
BACK
OFFICE
API API
BACK
OFFICE
API API
API
CRON
MAKE IT DETACHABLE

24. MICROSERVICE
MAKE IT DETACHABLE
BACK
OFFICE
API API
BACK
OFFICE
API API
API
CRON

25. FAST AND EASY TO ROLLBACK
DEVELOPMENT REQUIREMENT
WE NEED TO ROLLBACK
SERIOUSLY

26. WHY WHAT HOW

27. BEFORE WHAT

29. VM DOCKER
WHAT IS DOCKER?
@http://pyrasis.com/book/DockerForTheReallyImpatient/Chapter01/01

30. VM DOCKER
WHAT IS DOCKER?
@http://pyrasis.com/book/DockerForTheReallyImpatient/Chapter01/01

31. WHAT IS DOCKER?

32. WHAT IS DOCKER?
LIGHT VIRTUAL MACHINE
VMs
SERVER

33. IMAGE CONTAINER
WHAT IS DOCKER?
MEANWHILE IN DOCKER WORLD
Windows 8 PCs
@CPU by Creative Stall from the Noun Project

34. WHAT IS DOCKER?
IMAGE
BASE
UBUNTU, ALPINE, CENTOS …
DEPENDENCIES
VIM, CURL, NODE.JS, PYTHON …
APPLICATION
SOURCE CODE, ENVIRONMENT …
START SCRIPT
BASH, MAKEFILE, NPM START …

35. WHAT IS DOCKER?
CONTAINER
INFORMATION
NAME, PORT …
SETUP
RESTART, DAEMON, LINK …
HOST RELATED
VOLUME, NETWORK …
PROCESS
NODE …

36. WHY WHAT HOW

37. EASIER ENVIRONMENT SETUP
UNABLE TO VERSION CONTROL
$ curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
$ sudo apt-get update
$ sudo apt-get install nodejs
…
$ echo “deb http://nginx.org/packages/debian/ squeeze nginx” >> /etc/apt/sources.list
$ echo “deb-src http://nginx.org/packages/debian/ squeeze nginx” >> /etc/apt/sources.list
$ sudo apt-get install nginx
…
$ sudo apt-get install ntp vim fail2ban
…
$ vi /etc/nginx/site-enabled/node.conf
$ nginx -s reload
…
$ mkdir /home/alma/app && cd /home/alma/app
$ git pull … && npm install
$ npm run build
$ npm run serve

38. EASIER ENVIRONMENT SETUP
DOCKERIZED WAY
FROM node:6
RUN
apt-get update --force-yes -y &&
mkdir -p /src
COPY package.json /src/package.json
RUN
apt-get update --force-yes -y &&
apt-get install --force-yes -y vim
RUN
cd /src; npm install --production
COPY . /src
RUN
cd /src; npm run build
EXPOSE 3000
ENV NODE_ENV production
ENV DB live
ENV TZ Asia/Seoul
WORKDIR /src
CMD npm run serve
Dockerfile

39. EASIER ENVIRONMENT SETUP
DOCKERIZED WAY
$ docker build -t node-docker .
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
node-docker latest 08a5b1c92fcf About a minute ago 654.5 MB

40. EASIER ENVIRONMENT SETUP
OFFICIAL NODE Dockerfile
FROM buildpack-deps:jessie
RUN groupadd -r node && useradd -r -g node node
# gpg keys listed at https://github.com/nodejs/node
RUN set -ex
&& for key in
9554F04D7259F04124DE6B476D5A82AC7E37093B 94AE36675C464D64BAFA68DD7434390BDBE9B9C5
0034A06D9D9B0064CE8ADF6BF1747F4AD2306D93 FD3A5288F042B6850C66B31F09FE44734EB7990E
71DCFD284A79C3B38668286BC97EC7A07EDE3FC1 DD8F2338BAE7501E3DD5AC78C273792F7D83545D
B9AE9905FFD7803F25714661B63B535A4C206CA9 C4F0DFFF4E8C1A8236409D08E73BC641CC11F4C8
; do
gpg --keyserver ha.pool.sks-keyservers.net --recv-keys "$key";
done
ENV NPM_CONFIG_LOGLEVEL info
ENV NODE_VERSION 6.9.1
RUN curl -SLO "https://nodejs.org/dist/v$NODE_VERSION/node-v$NODE_VERSION-linux-x64.tar.xz"
&& curl -SLO "https://nodejs.org/dist/v$NODE_VERSION/SHASUMS256.txt.asc"
&& gpg --batch --decrypt --output SHASUMS256.txt SHASUMS256.txt.asc
&& grep " node-v$NODE_VERSION-linux-x64.tar.xz$" SHASUMS256.txt | sha256sum -c -
&& tar -xJf "node-v$NODE_VERSION-linux-x64.tar.xz" -C /usr/local --strip-components=1
&& rm "node-v$NODE_VERSION-linux-x64.tar.xz" SHASUMS256.txt.asc SHASUMS256.txt
&& ln -s /usr/local/bin/node /usr/local/bin/nodejs
CMD [ "node" ]
Dockerfile

41. EASIER ENVIRONMENT SETUP
DOCKERIZED WAY
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
node-docker latest 08a5b1c92fcf About a minute ago 654.5 MB
$ docker run --name docker-instance-1 --port 80:3000 node-docker

42. SINGLE-THREADED in MULTI-CORE
EASY TO SCALE
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
node-docker latest 08a5b1c92fcf About a minute ago 654.5 MB
$ docker run --name docker-instance-1 --port 80:3000 node-docker
$ docker run --name docker-instance-2 --port 81:3000 node-docker
$ docker run --name docker-instance-3 --port 82:3000 node-docker
.
.
.

43. SINGLE-THREADED in MULTI-CORE
EASY TO SCALE
http {
upstream application {
least_conn;
server localhost:80 max_fails=3 fail_timeout=60s;
server localhost:81 max_fails=3 fail_timeout=60s;
server localhost:82 max_fails=3 fail_timeout=60s;
}
server {
listen 80;
location / {
proxy_pass http://application;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}
}
nginx.conf

44. MICROSERVICE
DOCKER-COMPOSE
version: '2'
services:
application:
container_name: application
image: node-docker
docker-compose.yml

45. MICROSERVICE
DOCKER-COMPOSE
version: '2'
services:
application:
container_name: application
image: node-docker
redis:
container_name: redis
image: redis
docker-compose.yml

46. MICROSERVICE
DOCKER-COMPOSE
version: '2'
services:
application:
container_name: application
image: node-docker
redis:
container_name: redis
image: redis
nginx:
container_name: nginx
image: nginx
ports:
- 80:80
- 443:443
docker-compose.yml

47. MICROSERVICE
DOCKER-COMPOSE
version: '2'
services:
application:
container_name: application
image: node-docker
application-alt:
container_name: application
image: node-docker
redis:
container_name: redis
image: redis
nginx:
container_name: nginx
image: nginx
ports:
- 80:80
- 443:443
docker-compose.yml

48. MICROSERVICE
DOCKER-COMPOSE
version: '2'
services:
application:
container_name: application
image: node-docker
application-alt:
container_name: application
image: node-docker
redis:
container_name: redis
image: redis
nginx:
container_name: nginx
build: ./nginx
restart: always
links:
- application-1:application
- application-2:application-alt
ports:
- 80:80
- 443:443
depends_on:
- application
- application-alt
docker-compose.yml

49. FAST AND EASY TO ROLLBACK
BUILD WITH TAG
$ docker build -t node-docker .
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
node-docker latest 08a5b1c92fcf About an hour ago 654.5 M
$ docker build -t node-docker:1982 .
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
node-docker latest 08a5b1c92fcf About an hour ago 654.5 MB
node-docker 1982 10394kdfksf1 About a minute ago 654.5 MB

50. FAST AND EASY TO ROLLBACK
BUILD WITH TAG
$ docker build -t node-docker .
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
node-docker latest 08a5b1c92fcf About an hour ago 654.5 M
$ docker build -t node-docker:1982 .
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
node-docker latest 08a5b1c92fcf About an hour ago 654.5 MB
node-docker 1982 10394kdfksf1 About a minute ago 654.5 MB

51. FAST AND EASY TO ROLLBACK

52. FAST AND EASY TO ROLLBACK
DOCKER HUB
https://hub.docker.com/r/library/node/tags/

53. WHY WHAT HOW

54. YOU CAN DOCKERIZE
YOUR APPLICATION TOMORROW

55. $ npm init -f
$ npm i -S express uuid
DOCKERIZE
START WITH EXPRESS

56. DOCKERIZE
START WITH EXPRESS
var express = require('express');
var uuid = require('uuid');
var app = express();
var id = uuid.v4();
var port = 3000;
app.get('/', function (req, res) {
res.send(id)
});
app.listen(port, function () {
console.log('Example app listening on port: ' + port);
});
index.js

57. DOCKERIZE
BUILD DOCKER IMAGE
node_modules/
.dockerignore

58. DOCKERIZE
FROM node:6
COPY package.json /src/package.json
RUN cd /src; npm install
COPY . /src
EXPOSE 3000
WORKDIR /src
CMD node index.js
Dockerfile
BUILD DOCKER IMAGE

59. $ docker build --tag node-nginx:test .
Sending build context to Docker daemon 7.168 kB
Step 1 : FROM node:6
6: Pulling from library/node
357ea8c3d80b: Already exists
52befadefd24: Already exists
3c0732d5313c: Pull complete
ceb711c7e301: Pull complete
868b1d0e2aad: Pull complete
61d10f626f84: Pull complete
Digest: sha256:12899eea666e85f23e9850bd3c309b1ee28dd0869f554a7a6895fc962d9094a3
Status: Downloaded newer image for node:6
---> 800da22d0e7b
Step 2 : COPY package.json /src/package.json
---> 7f3344975b1e
Removing intermediate container ae1d0482e982
Step 3 : RUN cd /src; npm install --production
---> Running in 222a0585301b
...
Successfully built 08a5b1c92fcf
BUILD DOCKER IMAGE
DOCKERIZE

60. $ docker run --name node-nginx-instance -p 3000:3000 node-nginx:test
Example app listening on port: 3000
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS
NAMES
30179995521c node-nginx:test "/bin/sh -c 'node ind" About a minute ago Up About a minute 0.0.0.0:3000->3000/
tcp node-nginx-instance
BUILD DOCKER IMAGE
DOCKERIZE

62. $ docker run -d --name node-nginx-instance-0 -p 3000:3000 node-nginx:test
$ docker run -d --name node-nginx-instance-1 -p 3001:3000 node-nginx:test
$ docker run -d --name node-nginx-instance-2 -p 3002:3000 node-nginx:test
RUN 3 DOCKER CONTAINER
DOCKERIZE

63. worker_processes 1;
events { worker_connections 1024; }
http {
upstream node-app {
least_conn;
server localhost:3000;
server localhost:3001;
server localhost:3002;
}
server {
listen 80;
location / {
proxy_pass http://node-app;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}
}
LOAD-BALANCING WITH NGINX
nginx/nginx.conf
DOCKERIZE

64. FROM nginx
COPY nginx.conf /etc/nginx/nginx.conf
LOAD-BALANCING WITH NGINX
nginx/Dockerfile
DOCKERIZE

65. $ docker build --tag node-nginx-lb:test .
$ docker run -d --name node-nginx-instance-lb -p 4000:80 node-nginx-lb:test
LOAD-BALANCING WITH NGINX
DOCKERIZE

66. DOCKERIZE
NOOOOOOOOT WORKING
WHAT THE…

67. DOCKERIZE
I RUINED PLAYNODE

68. $ docker rm -f $(docker ps -a -q)
LINK CONTAINERS
DOCKERIZE

69. version: '2'
services:
nginx:
container_name: node-nginx-lb
build: ./nginx
links:
- app-1:app-1
- app-2:app-2
- app-3:app-3
ports:
- 3000:80
depends_on:
- app-1
- app-2
- app-3
app-1:
container_name: node-nginx-1
image: node-nginx:test
ports:
- 3000
app-2:
container_name: node-nginx-2
image: node-nginx:test
ports:
- 3000
LINK CONTAINERS
DOCKERIZE
app-3:
container_name: node-nginx-3
image: node-nginx:test
ports:
- 3000
docker-compose.yml

70. ...
upstream node-app {
least_conn;
server app-1:3000 max_fails=3 fail_timeout=30s;
server app-2:3000 max_fails=3 fail_timeout=30s;
server app-3:3000 max_fails=3 fail_timeout=30s;
}
...
LINK CONTAINERS
DOCKERIZE
nginx/nginx.conf

71. $ cd ..
$ docker-compose up
Recreating node-nginx-3
Recreating node-nginx-1
Recreating node-nginx-2
Recreating node-nginx-lb
Attaching to node-nginx-1, node-nginx-2, node-nginx-3, node-nginx-lb
node-nginx-1 | Example app listening on port: 3000
node-nginx-2 | Example app listening on port: 3000
node-nginx-3 | Example app listening on port: 3000
node-nginx-lb | 172.19.0.1 - - [22/Aug/2016:16:07:32 +0000] "GET / HTTP/1.1" 200 36 "-" "Mozilla/5.0 (Macintosh;
Intel Mac OS X 10_12) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Safari/602.1.50"
node-nginx-lb | 172.19.0.1 - - [22/Aug/2016:16:07:32 +0000] "GET / HTTP/1.1" 200 36 "-" "Mozilla/5.0 (Macintosh;
Intel Mac OS X 10_12) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Safari/602.1.50"
node-nginx-lb | 172.19.0.1 - - [22/Aug/2016:16:07:33 +0000] "GET / HTTP/1.1" 200 36 "-" "Mozilla/5.0 (Macintosh;
Intel Mac OS X 10_12) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Safari/602.1.50"
node-nginx-lb | 172.19.0.1 - - [22/Aug/2016:16:07:33 +0000] "GET / HTTP/1.1" 200 36 "-" "Mozilla/5.0 (Macintosh;
Intel Mac OS X 10_12) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Safari/602.1.50"
LINK CONTAINERS
DOCKERIZE

72. DOCKERIZE

73. AND THEN…

75. CLUSTERING
CONTAINER ORCHESTRATION
SWARM KUBERNATES

76. CLUSTERING
DOCKER SWARM

77. MONITORING
DOCKER STATS
$ docker stats determined_shockley determined_wozniak prickly_hypatia
CONTAINER CPU % MEM USAGE/LIMIT MEM % NET I/O
determined_shockley 0.00% 884 KiB/1.961 GiB 0.04% 648 B/648 B
determined_wozniak 0.00% 1.723 MiB/1.961 GiB 0.09% 1.266 KiB/648 B
prickly_hypatia 0.00% 740 KiB/1.961 GiB 0.04% 1.898 KiB/648 B

78. MONITORING
KITEMATIC FOR MAC (BETA)

79. MONITORING
DATADOG

80. CLOUD
DOCKER CLOUD

81. CLOUD
AWS ECS

82. – from Twitter
“Dockerize all the things!”

83. WE ARE HIRING
http://socar.recruiter.co.kr

84. alma@socar.kr
colus001@me.com
http://seokjun.kr