Python and Docker, How to easily set up the test environment using Docker?

1. K E E P I T S I M P L E ,
W E B D E V E L O P M E N T S TA C K
P Y C O N H K - 2 0 1 5
E R I C ( B Y U N G W O O K ) A H N
1

2. Who am I
Eric (ByungWook) Ahn
G+ : https://plus.google.com/+EricAhns
Experienced :
Device driver(windows, linux)
Media streaming
CDN
Docker
Platform Architecture Team at SK planet co., Ltd.

4. D O C K E R
S A M P L E A P P A R C H I T E C T U R E
T E S T E N V I R O N M E N T
P E R F O R M A N C E
T O D A Y …

5. h t t p s : / / w w w . d o c k e r . c o m /
B U I L D , S H I P, R U N
A n o p e n p l a t f o r m f o r
d i s t r i b u t e d a p p l i c a t i o n s f o r
d e v e l o p e r s a n d s y s a d m i n s
W H A T I S D O C K E R

6. C O N TA I N E R

7. F R O M u b u n t u
R U N
a p t - g e t u p d a t e & &
a p t - g e t i n s t a l l - y p y t h o n p y t h o n - d e v p y t h o n -
p i p p y t h o n - v i r t u a l e n v & &
r m - r f / v a r / l i b / a p t / l i s t s / *
$ c a t D o c k e r f i l e

8. $ docker build -t=“mypython” .
$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
mypython latest ecc926c495f8 7 seconds ago 373.6 MB
ubuntu latest 91e54dfb1179 4 days ago 188.4 MB
mysql latest c45e4ba02f47 12 days ago 283.8 MB
python 2.7 e1857ee1f3b5 5 weeks ago 674.4 MB
nginx latest 6886fb5a9b8d 5 weeks ago 132.9 MB
$ docker run -it mypython python
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b596713d6934 mypython "python" 2 seconds ago Exited (0) 2 seconds ago clever_hypatia
image create ( referenced at
Dockerfile of current directoy)
Execute a Container
Print a containers on docker hosts

9. Docker Host example
- docker container : ubuntu 2ea
- container : ubuntu(nginx)
$ top

10. D O C K E R - M A C H I N E VirtualBox
Microsoft Hyper-V
SOFTLAYER
openstack
Microsoft Azure
AWS
rackspace
DigitalOcean
create & remove
run
$ d o c k e r- m a c h i n e l s
N A M E A C T I V E D R I V E R S TAT E U R L S WA R M
d e f a u l t v i r t u a l b o x S t o p p e d
d e v v i r t u a l b o x S t o p p e d
p y c o n h k v i r t u a l b o x S t o p p e d
p y c o n h k 0 2 v i r t u a l b o x S t o p p e d
p y c o n h k 0 3 v i r t u a l b o x S t o p p e d
p y c o n h k 0 4 v i r t u a l b o x S t o p p e d
p y c o n h k 0 5 v i r t u a l b o x S t o p p e d
$ d o c k e r- m a c h i n e
$ d o c k e r

11. DOCKER-COMPOSE
W E B
D B
$ docker-compose up -d
C O N F I G U R AT I O N ?
= > YA M L
$ cat docker-compose.yml
db:
image: mysql
environment:
- MYSQL_ROOT_PASSWORD=hellopython
- MYSQL_DATABASE=shopping_db
ports:
- "3306:3306"
web:
build: ./web_api/
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/web_api
ports:
- "8000:8000"
links:
- db:db

12. {“product_code":"1004",
"product_title":"BLOOMSBURY BOUQUET WHEELED SUITCASE”,
"product_price":115}
Sample Application

13. Sample Application
A N G U L A R
J S O N
P Y T H O N
R E S T A P I
P Y T H O N
R E S T A P I
D B
L O A D
B A L A N C E R
L O A D
B A L A N C E R
A N G U L A R
J S O N

14. A N G U L A R
J S O N
P Y T H O N
R E S T A P I
P Y T H O N
R E S T A P I
D B
L O A D
B A L A N C E R
L O A D
B A L A C E R
A N G U L A R
J S O N
J AVA
R E S T A P I
J AVA
R E S T A P I
G O L A N G
R E S T A P I
G O L A N G
R E S T A P I
R O U T I N G
R E D I S
Restful API…

15. T E S T E N V I R O N M E N T
Traffic Generation Server
Dell R620
2.0 GHz/2P/12C
32G
Docker Host.
HP DL380G7
2.13GHz/1P/4C
24GB
load testing tool?
nGrinder/Apache ab/locust.io

16. T E S T E N V I R O N M E N T
$ docker inspect web1
[{
"AppArmorProfile": "",
"Args": [
"-g",
"daemon off;"
],
"Config": {
"AttachStderr": false,
"AttachStdin": false,
"AttachStdout": false,
"Cmd": [
"nginx",
"-g",
"daemon off;"
],
"CpuShares": 1024, =>
"Cpuset": "",
"Domainname": “",
…
…
$ docker inspect web2
[{
"AppArmorProfile": "",
"Args": [
"-g",
"daemon off;"
],
"Config": {
"AttachStderr": false,
"AttachStdin": false,
"AttachStdout": false,
"Cmd": [
"nginx",
"-g",
"daemon off;"
],
"CpuShares": 512, =>
"Cpuset": "",
"Domainname": “",
…
…
$ docker inspect web3
[{
"AppArmorProfile": "",
"Args": [
"-g",
"daemon off;"
],
"Config": {
"AttachStderr": false,
"AttachStdin": false,
"AttachStdout": false,
"Cmd": [
"nginx",
"-g",
"daemon off;"
],
"CpuShares": 512, =>
"Cpuset": "",
"Domainname": “",
…
…
C P U :
5 0 %
C P U :
2 5 %
C P U :
2 5 %

17. Browser
http://demo/
A P P L I C AT I O N S TA C K
A n g u l a r J S
A P I s : h t t p : / / w e b _ a p i : 8 0 0 0 / s h o p p i n g /
P y t h o n 1 . 7 . 0
D j a n g o 1 . 8 . 3
M y S Q L 5 . 6 . 2 3 , P o s t g re S Q L 9 . 4 . 4
m e m c a c h e d

18. T E S T C A S E
Django Cache Database -
Case_01 1.8.3 - Mysql 5.6.23 -
Case_02 1.8.3 - - -
Case_03 1.8.3 -
Postgresql
9.4.4
-
Case_04 1.8.3 Memcache Mysql 5.6.23 -
Demo 1.8.3 Memcache Mysql 5.6.23
Load
Balancer

19. N G R I N D E R / A PA C H E A B / L O C U S T. I O
nGrinder : create 1 controller and 5 agents container
// controller
$ docker run -d -v ~/.ngrinder:/root/.ngrinder --name ngrinder -p 80:80 -p
16001:16001 -p 12000-12009:12000-12009 ngrinder/controller:3.3
// agent : 5th
$ docker run -d -e 'CONTROLLER_ADDR=ngrinder:80' --link ngrinder:ngrinder
ngrinder/agent:3.3
$ docker run -d -e 'CONTROLLER_ADDR=ngrinder:80' --link ngrinder:ngrinder
ngrinder/agent:3.3
$ docker run -d -e 'CONTROLLER_ADDR=ngrinder:80' --link ngrinder:ngrinder
ngrinder/agent:3.3
$ docker run -d -e 'CONTROLLER_ADDR=ngrinder:80' --link ngrinder:ngrinder
ngrinder/agent:3.3
$ docker run -d -e 'CONTROLLER_ADDR=ngrinder:80' --link ngrinder:ngrinder
ngrinder/agent:3.3

20. container
nginx : 1.9.2
index.html, 600 bytescontainer x 5
nGrinder agent

21. D E M O - C A S E 1 : S A M P L E A P P
$ cat settings.py
INSTALLED_APPS = (
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'shopping'
)
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'shopping_db',
'USER': 'root',
'PASSWORD': 'hellopython',
'HOST': 'db',
'PORT': 3306,
}
}

22. S A M P L E A P P
models.py
from django.db import models
class Shopping(models.Model):
product_code = models.TextField()
product_title = models.CharField(max_length=100, verbose_name = "Product name")
product_price = models.IntegerField(default='0', verbose_name = "Product price")
class Meta:
verbose_name = "Product list"
verbose_name_plural = "Favorite lists"

23. S A M P L E A P P
// views.py
def shopping_list(request):
"""
List all code shoppinglists, or create a new shopping.
"""
if request.method == 'GET':
shoppinglists = Shopping.objects.all()
serializer = ShoppingSerializer(shoppinglists, many=True)
return JSONResponse(serializer.data)
// serializers.py
from rest_framework import serializers
from shopping.models import Shopping
class ShoppingSerializer(serializers.ModelSerializer):
class Meta:
model = Shopping
fields = ('product_code', 'product_title', 'product_price')
urls.py
urlpatterns = [
url(r'^shopping/$', views.shopping_list)
]

24. S A M P L E A P P
$ cat docker-compose.yml
db:
image: mysql
environment:
- MYSQL_ROOT_PASSWORD=hellopython
- MYSQL_DATABASE=shopping_db
ports:
- "3306:3306"
web:
build: ./web_api/
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/web_api
ports:
- "8000:8000"
links:
- db:db

25. demo : case1

26. P E R F O R M A N C E : C A S E 1
P Y T H O N
R E S T A P I
M Y S Q LL O A D T O O L

27. D E M O : C A S E 2
views.py
def shopping_list(request):
"""
List all code shoppinglists, or create a new shopping.
"""
if request.method == 'GET':
return JSONResponse({“product_code":"1005",
"product_title":"BILLIE GOES TO TOWN SHOPPER","product_price":28})

28. P E R F O R M A N C E : C A S E 2
P Y T H O N
R E S T A P I
L O A D T O O L

29. N G I N X - G U N I C O R N
$ cat dockdr-compose.yml
nginx:
restart: always
build: ./nginx/
ports:
- "80:80"
volumes:
- /www/static
volumes_from:
- web
links:
- web:web
web:
restart: always
build: ./web
expose:
- "8000"
volumes:
- /usr/src/app/static
env_file: .env
command: /usr/local/bin/gunicorn docker_django.wsgi:application -w 2 -b :8000
N G I N X
+ G U N I C O R N
P Y T H O N
R E S T A P I
L O A D T O O L

30. N G I N X - G U N I C O R N
$ cat nginx.conf
server {
listen 80;
server_name example.org;
charset utf-8;
location /static {
alias /usr/src/app/static;
}
location / {
proxy_pass http://web:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}

31. N G I N X - G U N I C O R N
gunicorn docker_django.wsgi:application -w 2

32. gunicorn docker_django.wsgi:application -w 12

33. D E M O : C A S E 3
$ cat settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'shopping_db',
'USER': 'postgres',
'PASSWORD': 'hellopython',
'HOST': 'db',
'PORT': 5432,
}
}

34. $ cat docker-compose.ml
web:
build: ./web_api/
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/web_api
ports:
- "8000:8000"
links:
- db:db
db:
image: postgres
ports:
- "5432:5432"
environment:
- DB_USER=postgres
- DB_PASS=hellopython
- DB_NAME=shopping_db
D E M O : C A S E 3

35. P E R F O R M A N C E : C A S E 3

36. D E M O : C A S E 4
settings.py
CACHE = {
'default':{
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': 'memcached:11211'
}
} views.py
def shopping_list(request):
"""
List all code shoppinglists, or create a new shopping.
"""
if request.method == 'GET':
data = cache.get('shoppinglists')
if data is None:
shoppinglists = Shopping.objects.all()
serializer = ShoppingSerializer(shoppinglists, many=True)
data = serializer.data
cache.set('shoppinglists', data, 60) # seconds
return JSONResponse(data)

37. $ cat docker-compose.yml
db:
image: mysql
environment:
- MYSQL_ROOT_PASSWORD=hellopython
- MYSQL_DATABASE=shopping_db
ports:
- "3306:3306"
memcached:
image: memcached:1.4
ports:
- "11211:11211"
D E M O : C A S E 4
web:
build: ./web_api/
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/web_api
ports:
- "8000:8000"
links:
- db:db
- memcached:memcached

38. P E R F O R M A N C E : C A S E 4
P Y T H O N
R E S T A P I
M Y S Q LL O A D T O O L
M E M C A C H E D

39. D E M O A R C H I T E C T U R E
P Y T H O N
R E S T A P I
M Y S Q L
M E M C A C H E D
W E B PA G E
H A P R O X Y
W E B PA G E
W E B PA G E
H A P R O X Y
P Y T H O N
R E S T A P I
P Y T H O N
R E S T A P I

40. demo-composer.yml
- haproxy
- mysql
- memcached
- python django restframeowork

41. web:
restart: always
build: ./web_api/
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/web_api
expose:
- "8000"
links:
- db
- memcached
weblb:
restart: always
image: tutum/haproxy
links:
- web
ports:
- "80:80"
environment:
- BACKEND_PORT=8000
$ docker-compose build .
$ docker-compose scale web=10
$ docker-compose up -d —no-recreate
db:
image: mysql
environment:
- MYSQL_ROOT_PASSWORD=hellopython
- MYSQL_DATABASE=shopping_db
ports:
- "3306"
memcached:
image: memcached:1.4
ports:
- "11211"

42. P E R F O R M A N C E : D E M O

43. DEMO

44. Conclusion
Python
Django
djangorestframawork
ORM
MySQL, Postgresql
memcached
Docker
Today reviewed :
flask
redis
memcached(memory-based)
other rest framework
gunicorn vs uwsgi
test environment
best practice
Next time:

45. Q&A