Mais conteúdo relacionado
Semelhante a Docker and Django Meet For A Tango - London Meetup (20)
Docker and Django Meet For A Tango - London Meetup
- 1. Copyright 2019 © systemati.co
Docker and Django
Meet For A Tango
Hendrik Frentrup
hendrik@systemati.co
15 July 2019
London, UK
- 2. Copyright 2019 © systemati.co
Ways to get in touch
hendrik@systemati.co
@HendrikFrentrup
linkedin.com/in/frentrup/
github.com/hendrikfrentrup
medium.com/@hendrik.frentrup
- 3. Copyright 2019 © systemati.co
Links
•GitHub repo for the code
•github.com/hendrikfrentrup/docker-django
•Walkthrough:
•medium.com/devopslinks/tech-edition-how-to-dockerize-a-django-web-
app-elegantly-924c0b83575d
•https://medium.com/@hendrik.frentrup/tech-edition-django-dockerization-
with-bells-and-whistles-and-a-tad-bit-of-cleverness-2b5d1b57e289
•Further reading:
•medium.com/panaseer-labs-engineering-data-science/how-devops-
helped-us-solve-our-big-data-development-problem-263474dfeedb
•Contact details:
•Email: hendrik@systemati.co
•Twitter: @hendrikfrentrup
•LinkedIn: linkedin.com/in/frentrup/
- 4. Copyright 2019 © systemati.co
Code!
• This will be a pretty detailed walk-through
• There will be a lot of code in detail
• Sometimes I’ll flick through things pretty quickly
• I’ll make the slides available
- 5. Copyright 2019 © systemati.co
The Problem
• Moving code from development environments to staging or
production can break things
• The state of environments drifts and is hard to manage
• For lack of access, developers have no insight into the specifics of
production environments
- 6. Copyright 2019 © systemati.co
Why?
• Ease the pain of moving from Dev to Prod
• Step towards being Cloud Native
• Twelve Factor App (https://12factor.net/)
• Infrastructure-as-Code
• Productivity
- 8. Copyright 2019 © systemati.co
To here
Docker
web
gunicorn
server
db
postgres
Local machine
db-admin
pgadmin
server
nginx
nginx
server
python
manage.py
runserver
*.conf
+ static/
pg-data/
- 9. Copyright 2019 © systemati.co
• Move a locally running app into a container
• Setup database, admin and nginx services
• Run migrations
• Logging
• Implement Infrastructure-as-code
• Bring development and production
environments close together
• Increase productivity & working collaboratively
Outline
- 10. Copyright 2019 © systemati.co
Simple app running locally
python manage.py runserver
db.sqlite3
Local machine
- 11. Copyright 2019 © systemati.co
10 commands to set up
> git init
> git checkout -b develop
> virtualenv env
> source env/bin/activate
> pip install Django>=2.1
> echo 'Django>=2.1' >> requirements.txt
> django-admin startproject my-site
> git add * && git commit -m "..."
> python mysite/manage.py runserver
- 12. Copyright 2019 © systemati.co
App running in container
Docker
web
python manage.py runserver
db.sqlite3
Local machine
- 13. Copyright 2019 © systemati.co
App inside Docker
> touch Dockerfile
> docker build -t django-docker:0.0.1 .
> docker run -p 8001:8001 docker-django:0.0.1
______Dockerfile____________________________________________________________
FROM python:3.6.7-alpine
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
ADD ./ /code/
RUN pip install -r requirements.txt
CMD ["python", "mysite/manage.py", "runserver", "0.0.0.0:8001"]
- 15. Copyright 2019 © systemati.co
Local storage
Docker
web
python manage.py runserver
db.sqlite3db.sqlite3
Local machine
- 16. Copyright 2019 © systemati.co
Moving to docker-compose
> touch docker-compose.yml
> docker-compose up
______docker-compose.yml__________________________________________________
version: "3"
services:
web:
build: .
command: python mysite/manage.py runserver 8001
ports:
- “8001:8001”
volumes:
- ./mysite/:/code
maps ports and container code/db to local filesystem
- 17. Copyright 2019 © systemati.co
Database service
Docker
web
python
manage.py
runserver
db
postgres
Local machine
- 18. Copyright 2019 © systemati.co
Adding a database service
______docker-compose.yml__________________________________________________
version: "3"
services:
web:
build: .
command: python mysite/manage.py runserver 8001
ports:
- “8001:8001”
volumes:
- ./mysite/:/code
db:
image: postgres
networks:
- backend
networks:
backend:
driver: bridge
- 19. Copyright 2019 © systemati.co
Handling env. variables
______settings.py____________________________________________________________
POSTGRES_PASSWORD = os.environ.get('POSTGRES_PASSWORD')
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'USER': 'postgres',
'PASSWORD': POSTGRES_PASSWORD,
'HOST': ‘db',
'PORT': '5432',
},
______.env__________________________________________________________________
POSTGRES_PASSWORD=secret
POSTGRES_USER=postgres
DJANGO_SECRET_KEY=secret
> touch .env
> echo 'psycopg2-binary' >> requirements.txt
- 20. Copyright 2019 © systemati.co
Credentials in envvars
______docker-compose.yml__________________________________________________
version: "3"
services:
web:
[...]
environment:
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_USER: ${POSTGRES_USER}
db:
[...]
environment:
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_USER: ${POSTGRES_USER}
- 21. Copyright 2019 © systemati.co
Rebuilding the web image
______Dockerfile____________________________________________________________
FROM python:3.6.7-alpine
ENV PYTHONUNBUFFERED 1
RUN apk update &&
apk add --virtual build-deps gcc python-dev musl-dev &&
apk add postgresql-dev
RUN mkdir /code
WORKDIR /code
ADD ./ /code/
RUN pip install -r requirements.txt
- 22. Copyright 2019 © systemati.co
A bit of admin
> docker-compose exec web python mysite/manage.py migrate
> docker-compose exec web python mysite/manage.py createsuperuser
Run that every time we tear down our containers?
Note down the credentials we’ve used?
- 23. Copyright 2019 © systemati.co
Data migrations
______0001_initial.py___________________________________________________________
import os
from django.db import migrations
class Migration(migrations.Migration):
dependencies = []
def generate_superuser(apps, schema_editor):
from django.contrib.auth.models import User
DJANGO_SU_NAME = os.environ.get('DJANGO_SU_NAME')
DJANGO_SU_EMAIL = os.environ.get('DJANGO_SU_EMAIL')
DJANGO_SU_PASSWORD = os.environ.get('DJANGO_SU_PASSWORD')
superuser = User.objects.create_superuser(
username=DJANGO_SU_NAME, email=DJANGO_SU_EMAIL,
password=DJANGO_SU_PASSWORD)
superuser.save()
operations = [
migrations.RunPython(generate_superuser),
]
- 24. Copyright 2019 © systemati.co
More credentials
______docker-compose.yml__________________________________________________
version: "3"
services:
web:
[...]
environment:
DJANGO_DB_NAME: ${DJANGO_DB_NAME}
DJANGO_SU_NAME: ${DJANGO_SU_NAME}
DJANGO_SU_EMAIL: ${DJANGO_SU_EMAIL}
DJANGO_SU_PASSWORD: ${DJANGO_SU_PASSWORD}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_USER: ${POSTGRES_USER}
- 25. Copyright 2019 © systemati.co
web, db and admin services
Docker
web
python
manage.py
runserver
db
postgres
Local machine
db-admin
pgadmin server
- 26. Copyright 2019 © systemati.co
Adding a DB admin service
______docker-compose.yml__________________________________________________
[...]
db-admin:
image: dpage/pgadmin4
environment:
PGADMIN_DEFAULT_EMAIL: ${PGADMIN_DEFAULT_EMAIL}
PGADMIN_DEFAULT_PASSWORD: ${PGADMIN_DEFAULT_PASSWORD}
depends_on:
- db
ports:
- "8080:80"
networks:
- backend
[...]
- 28. Copyright 2019 © systemati.co
Local app for debugging
Docker
web
python
manage.py
runserver
db
postgres
Local machine
db-admin
pgadmin
server
python
manage.py
runserver
- 29. Copyright 2019 © systemati.co
Run a local app alongside
______settings.py____________________________________________________________
from decouple import config
SECRET_KEY = config(‘DJANGO_SECRET_KEY')
DEBUG = config('DJANGO_DEBUG', default=False, cast=bool)
WEB_HOST = config('DJANGO_WEB_HOST', default='localhost')
DB_HOST = config('DJANGO_DB_HOST', default='localhost')
DB_NAME = config(‘DJANGO_DB_NAME’)
[...]
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'USER': POSTGRES_USER,
'PASSWORD': POSTGRES_PASSWORD,
'HOST': DB_HOST,
'PORT': '5432',
},
- 30. Copyright 2019 © systemati.co
Run a local app alongside
______docker-compose.yml_____________________________________
[...]
environment:
DJANGO_DEBUG: "True"
DJANGO_SECRET_KEY: ${DJANGO_SECRET_KEY}
DJANGO_DB_HOST: “db"
[...]
- 31. Copyright 2019 © systemati.co
With nginx
Docker
web
gunicorn
server
db
postgres
Local machine
db-admin
pgadmin
server
nginx
nginx
server
python
manage.py
runserver
- 32. Copyright 2019 © systemati.co
Add nginx server
______default.conf_______________________________
server {
listen 80;
server_name localhost;
location / {
proxy_pass http://web:8001;
}
}
______settings.py________________________________
[...]
ALLOWED_HOSTS = [WEB_HOST, ‘localhost']
[...]
> echo “gunicorn” >> requirements.txt
- 33. Copyright 2019 © systemati.co
Add nginx server
______docker-compose.yml__________________________________________________
[...]
web:
command: gunicorn mysite.wsgi --bind 0.0.0.0:8001
[...]
nginx:
image: nginx:latest
ports:
- 8088:80
volumes:
- ./nginx/default.conf:/etc/nginx/conf.d/default.conf
networks:
- backend
depends_on:
- web
[...]
- 34. Copyright 2019 © systemati.co
Static files & persistent data
Docker
web
gunicorn
server
db
postgres
Local machine
db-admin
pgadmin
server
nginx
nginx
server
python
manage.py
runserver
*.conf
+ static/
pg-data/
- 35. Copyright 2019 © systemati.co
Serving static assets
______settings.py_______________________________________
[...]
STATIC_ROOT = os.path.join(BASE_DIR, 'static')
> python mysite/manage.py collectstatic
______default.conf_____________________
server {
[...]
location /static {
alias /code/static;
}
}
- 36. Copyright 2019 © systemati.co
Serving static assets
______docker-compose.yml__________________________________________________
[...]
nginx:
[...]
volumes:
- ./nginx/default.conf:/etc/nginx/conf.d/default.conf
- ./mysite/static:/code/static
[...]
db:
volumes:
- ./postgres-data:/var/lib/postgresql/data
[...]
- 37. Copyright 2019 © systemati.co
Mapped ports
Docker
web
8001
db
5432
Local machine
db-admin
8080
nginx
8088
8000
- 38. Copyright 2019 © systemati.co
Splitting YAML files
______docker-compose.override.yml_____________
[...]
volumes:
- ./mysite:/code
environment:
DJANGO_DEBUG: “True"
[...]
______docker-compose.yml_______________
[...]
web:
build: .
command: gunicorn mysite.wsgi
--bind 0.0.0.0:8001
depends_on:
- db
networks:
- backend
[...]
- 41. Copyright 2019 © systemati.co
Logs in shared envs
• Complex topic and no one size fits all
• Log aggregator (SumoLogic, Splunk, ELK etc.)
• A complete ELK stack in one container
• Various logging driver options
• Fluentd
• Gelf
Log aggregatorFluentd
LogStash
ElasticSearch
Kibana
- 42. Copyright 2019 © systemati.co
Logging
Local machine
Staging environment
Build
server
Log aggregator
Production
Code
repo
Local machine
push
trigger
tail
deploy
stream
deploy
stream