Baking docker using chef

 Ruby, Java, C Developer turned into DevOps Architect
 Contributed to Chef development
 Chef azure extension
 Knife plugins: knife-azure, knife-ec2, knife-openstack
 Knife WinRM, knife windows listener
 Working with iHealth Technologies
 Technology, innovation and the thirst to keep learning are what define me
 Love to travel, read, write
 Above all, I am a mother to two boys!
 @muktaa

Docker
Chef + Docker
CD pipeline that uses knife-ssh
Push Jobs
Chef Cookbook
Chef Containers
Our Story

Linux Container
3 Components:
Docker Engine
Docker Hub
Docker Images
Benefits:
Speed
Portability
Density
Open Source
“Can create lightweight,
self sufficient containers
from any application”

FROM ubuntu:14.04
RUN apt-get update
RUN apt-get install
libfuse-dev
ADD
dev.conf/etc/myapp-
config/
Codify your configuration
Set of bash commands
Example:
HelloScala
Dockerfile
dev.conf
Docker build HelloScala

Shared Hosting – PaaS
Microservices
Lightweight Testing

Getting the best of both worlds!

Automate Make Whole Enchilada Deliver!

Control the environment Vs System Image /
Runtime image
Tradeoff between flexibility and manageability
CM is the vein of DevOps
Shell scripts -> Chef
Immutable Infrastructure

Replaces Human Tasks,
Idempotence,
Thick client - thin servers,
Order Matters,
Huge Community Support
An improved Robot,
Fast,
Easy,
Fresh fish in the market,
ready to be baked!

Because simple things can bring the most happiness!

•git push
•Triggers
Build
Code
•Build tools
have docker
support
•Build tools
generate a
docker
image
Build
Process Save imageDocker
Image Unique tagDocker
Registry
•docker pull
•docker stop
•docker run
Deploy
using knife-
ssh or Push
Jobs
CI Server

 git push to https://github.com/muktaa/HelloScala
 Triggers a build on your CI server
 sbt docker
 docker push muktaa/hello-scala
 knife ssh 'role:test' 'deploy.sh' -x ssh-user -i ssh-key -c knife.rb
 Build tools offer docker integration
 Eg: Maven has docker-maven-plugin
 https://github.com/spotify/docker-maven-plugin
 mvn clean package docker:build -DpushImage

~/github/HelloScala > sbt docker
[info] Loading project definition from
/Users/muktaaphale/github/HelloScala/project
[info] Set current project to hello-scala (in build
file:/Users/muktaaphale/github/HelloScala/)
[info] Creating docker image with name: 'muktaa/hello-scala'
:
[info] Sending build context to Docker daemon
[info] Step 0 : FROM dockerfile/java
[info] ---> 1126c85d8a06
[info] Step 1 : ADD /app/hello-scala_2.11-1.4-one-jar.jar
/app/hello-scala_2.11-1.4-one-jar.jar
[info] ---> Using cache
[info] ---> 61871958f108
[info] Step 2 : ENTRYPOINT java -jar /app/hello-scala_2.11-1.4-
one-jar.jar
[info] ---> Using cache
[info] ---> a8005b32ddc4
[info] Successfully built a8005b32ddc4
[info] Successfully built Docker image: muktaa/hello-scala
[success] Total time: 1 s, completed Mar 3, 2015 2:10:04 PM
~/github/HelloScala > docker images | grep hello-scala
muktaa/hello-scala latest a8005b32ddc4 12 hours ago
715 MB
~/github/HelloScala > docker run muktaa/hello-scala
Hello, world! #1
Hello, world! #2
Hello, world! #3

Docker Hub
Link:
https://registry.hub.docker.com/u/m
uktaa/hello-scala
Automated Build in Docker:
https://registry.hub.docker.com/u/m
uktaa/helloscala-automated-build/

Knife-ssh works like “push”. Almost.
Journey from pull to push
“Chef push jobs is an extension of the Chef server that allows
jobs to be run against nodes independently of a chef-client
run”
Job: set of commands to be run on node
 Docker pull
 Docker stop
 Docker run

Push Jobs
 Use message bus (zeromq)
 Claims to attack the scalability
issue
 Deployment status is relayed
back
 New born baby
 Complex at the moment, ready
with just the basic foundation
Knife SSH
 Parallel ssh
 SSH Protocol is slow and CPU
hungry at scale
 Feedback on deployment status
is not as easy
 Been in the market for long
 Easy to use

Enterprise Chef 11 or Chef server 12
Standalone or HA
Run the commands on Chef Server:
chef-server-ctl install opscode-push-jobs-server
opscode-push-jobs-server-ctl reconfigure
chef-server-ctl reconfigure

 Install knife push plugin
 Gem install knife-jobs
 Knife cookbook site download push-jobs
 Extract and save to your cookbook path
 Edit the attributes file (push-jobs/attributes/default.rb)
 default['push_jobs']['package_url'] = 'https://opscode-private-
chef.s3.amazonaws.com/ubuntu/12.04/x86_64/opscode-push-jobs-
client_1.1.5-1_amd64.deb'
 default['push_jobs']['package_checksum'] =
'd659c06c72397ed2bc6cd88488349857f1958538‘
 Upload the push-jobs cookbook to your ChefServer

Create 2 groups
 Pushy_job_writers
 Pushy_job_readers
Add user to the groups
Sudo chef-client –r “recipe[push-jobs]”
From Workstation:
 Knife node status
 Knife node status <node-name>

 knife job start ‘chef-client –r recipe[run-docker]’ <node-name>
knife job start ‘docker.sh’ my_node
Where docker.sh:
 Docker pull muktaa/hello-scala
 docker ps | grep muktaa/hello-scala| awk -F" " '{print $1}‘
 Docker run muktaa/hello-scala

If only applications were Hello World programs!

Application Configuration
Docker
Image

Packages Custom SetupsCredentials
Softwares Database
Files
Environment
Specific
Configuration
Ports

DEV
Docker
Container
Docker
Container
Docker
Container
PRE
PROD
Docker
Container
Docker
Container
Docker
Container
PROD
Docker
Container
Docker
Container
Docker
Container

Unsolved problem with Docker today
Credentials inside docker containers
Hard codes
Set environment variables

Create Base Image Manually,
with configuration embedded
Build Tool uses the custom
Base Image
Deploy using knife-ssh

To manage docker images and deployment

 Available in Supermarket: https://supermarket.chef.io/cookbooks/docker
 Install docker
 Build docker image
 Pull image and run container
 Push docker image to registry
 LWRPs
 Docker_container
 Docker_image
 Docker_registry
 https://github.com/bflad/chef-docker/blob/master/README.md

secret = Chef::EncryptedDataBagItem.load_secret
@docker_cred = Chef::EncryptedDataBagItem.load(
node['docker']['creds']['databag'],
node['docker']['user'],
secret
)
docker_registry ‘https://registry.hub.docker.com/u/muktaa/hello-scala/’ do
email docker_cred['email']
username docker_cred['username']
password docker_cred['password']
end

# Build a docker image using docker_image
resource
docker_image node['docker']['image'] do
tag node['docker']['image']['tag']
source '/var/docker'
action :build
end
# Push the image to docker registery
action :push
end
# Delete the image from the machine
action :remove
end

# Run Container
docker_container ‘muktaa/hello-scala’
detach true
port ‘8081:8081’, ‘8085:8085’
env ‘ENVIRONMENT=pre-prod’
volume ‘/mnt/docker/docker-storage’
action :run
end

# Generate a docker file using template.
template "#{node['docker']['directory']}/Dockerfile" do
source 'dockerfile.erb'
variables image: node['docker']['base']['image']['name'],
maintainer: @docker_cred['maintainer'],
email: docker_cred['email'],
build_cmd: node['docker']['build']['commands'],
entry_point: node['docker']['build']['entry_point']
action :create
end

Build
Application
• Save the Artifact to a Repository
Manager
Build
Docker
Image
• Docker cookbook would build and save
the docker image
Deploy
• Docker cookbook runs
the container on the
nodes

Package
Provides Configuration Management
for containers

Bootstrap chef-client without SSH
connection
Manage multiple services inside your
container
Manage running state of your container
Consistency across Architectures
Mixed Architecture Applications

Transitioning traditional architecture to containers
Handling last mile configuration when container boots
Getting the best of two worlds without complexity

Gem install knife-container
knife container docker init
NAMESPACE/IMAGE_NAME [options]
-f base docker image (default is ubuntu 12.04) - chef
container should be already installed on it
-r runlist
-z chef client local mode
-b use berkshelf

$ sudo knife container docker init muktaa/hello-scala-cc
Compiling Cookbooks...
Recipe: knife_container::docker_init
* directory[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc] action create
* template[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/Dockerfile] action
create
- update content in file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-
cc/Dockerfile from none to 943017
- * template[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/.dockerignore]
action create
- create new file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-
cc/.dockerignore
cc/.dockerignore from none to e3b0c4
* directory[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef] action
create
- create new directory /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef
* template[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef/client.rb]
action create
cc/chef/client.rb
cc/chef/client.rb from none to 7de61f
* file[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef/first-boot.json]
action create
- create new file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef/first-
boot.json
cc/chef/first-boot.json from none to 5269ef
* template[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef/.node_name]
action create
cc/chef/.node_name
cc/chef/.node_name from none to 4764d2
* template[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/Berksfile] action
create (skipped due to only_if)
* directory[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef/secure]
action create
- create new directory /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-
cc/chef/secure
* file[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-
cc/chef/secure/validation.pem] action create
cc/chef/secure/validation.pem
cc/chef/secure/validation.pem from none to ec1f3e
- change mode from '' to '0600'
Downloading base image: chef/ubuntu-12.04:latest. This process may take awhile...
Tagging base image chef/ubuntu-12.04 as muktaa/hello-scala-cc
Context Created: /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc

run command docker images
knife container docker build
resolve docker dependencies
build docker image
cleanup chef artifacts

$ sudo knife container docker build muktaa/hello-scala-cc
Sending build context to Docker daemon 9.728 kB
Sending build context to Docker daemon
Step 0 : FROM muktaa/hello-scala-cc
---> 50d3c5c9e133
Step 1 : ADD chef/ /etc/chef/
---> 4933cc9e13e0
Removing intermediate container da0a08413a91
Step 2 : RUN chef-init --bootstrap
---> Running in add27db609cc
[2015-03-31T21:44:44+00:00] INFO: Starting Supervisor...
[2015-03-31T21:44:44+00:00] INFO: Supervisor pid: 9
[2015-03-31T21:44:49+00:00] INFO: Starting chef-client run...
[2015-03-31T21:44:50+00:00] INFO: Forking chef instance to converge...
[2015-03-31T21:44:50+00:00] INFO: *** Chef 11.16.2 ***
[2015-03-31T21:44:50+00:00] INFO: Chef-client pid: 16
[2015-03-31T21:44:53+00:00] INFO: Client key /etc/chef/secure/client.pem is not present -
registering
[2015-03-31T21:44:53+00:00] INFO: HTTP Request Returned 404 Object Not Found: error
[2015-03-31T21:44:54+00:00] INFO: Setting the run_list to [] from CLI options
[2015-03-31T21:44:54+00:00] INFO: Run List is []
[2015-03-31T21:44:54+00:00] INFO: Run List expands to []
[2015-03-31T21:44:54+00:00] INFO: Starting Chef Run for muktaa-hello-scala-cc-build
[2015-03-31T21:44:54+00:00] INFO: Running start handlers
[2015-03-31T21:44:54+00:00] INFO: Start handlers complete.
[2015-03-31T21:44:55+00:00] INFO: Loading cookbooks []
[2015-03-31T21:44:55+00:00] WARN: Node muktaa-hello-scala-cc-build has an empty run list.
[2015-03-31T21:44:55+00:00] INFO: Chef Run complete in 1.121705004 seconds
[2015-03-31T21:44:55+00:00] INFO: Running report handlers
[2015-03-31T21:44:55+00:00] INFO: Report handlers complete
[2015-03-31T21:44:55+00:00] INFO: Sending resource update report (run-id: 6f637baf-18cc-
4620-b3e2-9afc90e8cd6b)
---> 2c2ec6fab1ef
Removing intermediate container add27db609cc
Step 3 : RUN rm -rf /etc/chef/secure/*
---> Running in 30a3611b083f
---> cab28d6eed90
Removing intermediate container 30a3611b083f
Step 4 : ENTRYPOINT ["chef-init"]
---> Running in 0a9f4e96bbf7
---> a8577b66b103
Removing intermediate container 0a9f4e96bbf7
Step 5 : CMD ["--onboot"]
---> Running in f9a444817229
---> 21b3800bc9b3
Removing intermediate container f9a444817229
Successfully built 21b3800bc9b3

$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL
SIZE
muktaa/hello-scala-cc latest 21b3800bc9b3 2 hours ago 311.9 MB
<none> <none> b343c8301cc8 2 hours ago 311.9 MB
chef/ubuntu-12.04 latest 50d3c5c9e133 6 months ago 311.9 MB
$ sudo docker push muktaa/hello-scala-cc
$ sudo docker –d run muktaa/hello-scala-cc

Product under Development. Super Cool DevOps Culture.

Running apps in containers is easy
Debugging apps in containers is difficult
You can very well run multiple services inside a
docker container
Ah the woes of Docker networking!
Sequential Progression
Bake carefully… Happy Baking!

Baking docker using chef

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Baking docker using chef

Semelhante a Baking docker using chef (20)

Mais de Mukta Aphale

Mais de Mukta Aphale (9)

Último

Último (20)

Baking docker using chef

Notas do Editor