Continuous Integration and Delivery into
Kubernetes with GitHub, TravisCI, and
Artifactory.
Google Cloud Platform
Continuous Integration and Delivery into
Kubernetes with GitHub, TravisCI, and
Artifactory
Google Cloud Platform
Oleg Shalygin
SADA Systems

Continuous Integration (CI) is a development practice that requires developers to integrate
code into a shared repository several times a day. Each check-in is then verified by an
automated build, allowing teams to detect problems early.
By adopting both Continuous Integration and Continuous Deployment, you not only reduce risks and
catch bugs quickly, but also move rapidly to working software.
https://www.thoughtworks.com/continuous-integration

GCP Free Tier
https://cloud.google.com/free/
○ 12 Months : $300
○ GAE
○ DataStore
○ Compute Engine
○ Container Engine
○ Cloud Storage
○ Pub/Sub
○ Cloud Functions(beta!)
○ Stackdriver
○ BigQuery

Registering with GCP
https://console.cloud.google.com
○ Login with your Google account
○ Agree to the Terms
○ Add a Credit Card
○ Start the Trial!

Google Cloud SDK
● Download the Google Cloud SDK
https://cloud.google.com/sdk/
○ Python 2.7 required
○ Download the tarball
○ Extract the package
○ $ ./google-cloud-sdk/install.sh
○ $ gcloud init

Proprietary + Confidential
GCP Essentials if you
when not using
gcloud init
$ gcloud auth login
# List what the current settings are
$ gcloud projects list
$ gcloud config list
# Set the project and compute zone
$ gcloud config set project
<YOUR_PROJECTNAME>
$ gcloud config set compute/zone <ZONE>

Cloud Load
Balancing
Web
Clients
Android
iOS
1
Relational Database
Cloud SQL
Async Messaging
Cloud Pub/Sub
Analytics
BigQuery
Image Bucket
Cloud Storage
Source Control
GitHub
Application Infrastructure
Monitoring
Stackdriver
Docker image storage
Container Registry
Container Engine
50GB PD
1 10.61
NoSQL Database
DataStore
Real Time Messaging
Firebase

Google Container Engine
Node
● Worker Machine that runs the
containers
● Managed by master components
● Included services:
○ Container runtime
○ kubelet
○ kube-proxy

Google Container Engine
● API centric system.
○ No private APIs
○ No system only calls
○ EVERYTHING
communicates through
the API.
● REST API in terms of
resources/verbs.
API Server

Google Container Engine
● Fundamental to how Kubernetes works
● Piece of code that watches the Kubernetes
API and reacts accordingly.
○ DNS, Kubelet, Services, ReplicaSet
Controllers

Google Container Engine
● Small group of containers and
volumes
● Tight coupling - live and die
together.
● Shared networking and lifecycle
○ Containers within a pod
share an IP address, they find
each other via localhost
Pods

Google Container Engine
● Declarative updates for pods.
● Specify a desired state =>
Deployment controller handles the
rest.
● Self healing
Deployments

Google Container Engine
● Abstraction that defines a logical
set of Pods and the policy by
which they are accessed.
● Recall that every pod has an IP.
● Services provide a stable VIP
● The pods behind the service can
scale up/down, be removed, etc,
their IP is constantly changing but
the service VIP does not change.
Services

Google Container Engine
● Holds sensitive information such as passwords,
OAuth tokens, encrypted files, etc.
● Numerous automatic secrets are created by
Kubernetes to store credentials to access the API.
● Secrets are base64 encoded(not encrypted)
● ConfigMaps are used to store non-sensitive
information.
● imagePullSecrets are used to set container registry
credentials.
Secrets / ConfigMaps

Google Container Engine
● CLI that allows you to run commands against the
cluster.
○ List pods
○ List nodes
○ Apply changes
○ “matrix” into a container on a running pod:
$ kubectl exec -it my-pod -- bash
● Enabled via:
$ gcloud components install kubectl
kubectl

Google Cloud Platform
● Simplify and automate cloud deployment
via CLI.
○ Repeatable
○ Codified
○ Declarative
● Templates can be pulled once created
manually for resource re-creation
GCP Deployment Manager

Google Cloud Platform
● Create a service account that is
used to manage the cluster and
orchestrate all of the commands
that we need.
● Used in the orchestration process.
● Located under IAM(Identity
Account Management)
● Encrypt and store as part of the CI
process
GCP Service Accounts

Travis CI
● Download and install the latest GCloud
SDK
○ https://cloud.google.com/sdk/do
wnloads
● Set the Zone, Cluster, and Project
● Activate the Service Account
● Install the kubectl CLI
Scripting out GCP Authentication

Build Orchestration
● Application code is decoupled from the
infrastructure code.
● Orchestrating deployments on various
environments or specific containers is completely
disassociated from the application code.
● Process Flow:
○ Application code is built, compiled or
packaged.
○ Code is pushed to the binary repository
which houses binaries of all sorts, such as:
npm modules, Java Archive (.jar), or
Docker images.
○ Upon deployment, the binary is pulled from
Artifactory and consumed as part of
another build orchestration step or
deployed to the appropriate resource.
● Modularity and flexibility!
Source Code built
(.jar, .js, .dll, images)
Infrastructure
Changes
Orchestrated
Deployed

Artifactory on GCP
● During the build, create a docker image.
● The image is pushed to Artifactory / GCR
○ There is native support within Kubernetes
on GKE with GCR.
○ In the case of a private repository such as
Artifactory, we need to create a
imagePullSecrets link via a Secrets config
type: kubernetes.io/dockerconfigjson
● Artifactory provides a free plan for open source
projects which allows users to ramp up and
experience the benefits of a binary repository:
https://www.jfrog.com/artifactory/free-trial/
Pushing Images from the build

Updating the Deployment
● If the build is successful, update the docker
image tag.
● Recall that the deployment specifies the
desired state of the pod.
Update container image through pipeline
# ...
spec:
replicas: 2
strategy:
rollingUpdate:
maxUnavailable: 0
type: RollingUpdate
template:
metadata:
labels:
app: md-web
spec:
containers:
- name: md-web
image: ‘us-gcr.io/project-name/web-applciation:1.9.1’
env:
- name: MONGO_IP
valueFrom:
secretKeyRef:
name: env-variables-secret
key: mongo_ip
- name: MONGO_PORT
valueFrom:
secretKeyRef:
name: env-variables-secret
key: mongo_port
imagePullPolicy: Always
ports:
# ...

Apply Changes
● Depending on how many deployments
describe the new state of the
application, the following command will
apply the changes to the cluster:
$ kubectl apply -f ./
Applying changes to the cluster

Artifactory - Universal Artifact Repository
● Software package binary repository
● Supports software packages created in any
language or package.
● CI/CD integration with nearly every CI provider
● Hook Properties for simple querying
● Atom Uses:
○ Docker
○ NPM
○ Maven
○ Gradle
○ Debian
○ Generic Repository

SaaS based
Asset and Resource
Management
Application

Cloud Load
Balancing
Web
Clients
Android
iOS
Container Engine
- GB PD
1
Redis
Compute Engine
- GB PD
1
Compute Engine
- GB PD
1 1
MongoDB
Compute Engine
- GB PD
2 1
Compute Engine
- GB PD
1 1
Relational DB #3
Cloud SQL
Relational DB #2
Cloud SQL
Relational DB #1
Cloud SQL
Async Messaging
Cloud Pub/Sub
User Account Management
Stormpath
Client AD
Reporting & Audit
Cloud Bigtable
Analytics
BigQuery
Image Bucket
Cloud Storage
Audit Bucket
Cloud Storage
Binary Repository
Artifactory
Source Control
GitHub
Atom Infrastructure
Monitoring
Stackdriver

Application Source Code (FrontEnd) - master branch
GitHub
Web Pod - DEV branch
GitHub
1. Build checks out the DEV branch in the associated pod repository and
updates a deployment container image.
2. The changes to deployment.yaml are committed to the pod repository
in the DEV branch.
Successful build generates a new image and pushes it to Artifactory.
1. Pull request created.
2. Pull request is approved.
3. Code is merged into master with a new application version.
Binary Repository
Artifactory
Web Pod Build - DEV branch
Travis CI
1. GCloud SDK, GCloud Authentication and build scripts are pulled from
Artifactory.
2. Successful build deploys the resource configurations to the Kubernetes cluster.
The new commit triggers a build on the master branch in Travis CI
Web Project Build - DEV branch
Travis CI
The new commit triggers a build on the DEV branch in Travis CI.
1. The images specified in the deployment are pulled from Artifactory.
2. A rolling update updates the underlying resources.
Continuous Integration - DEV Pipeline

Backend Pod - DEV branch
GitHub
Backend Pod Build - QA branch
Travis CI
1. A manual pull request is opened against the QA branch.
2. The pull request updates a single/multiple image tags
in the deployment.yaml to the versions in DEV.
3. The pull request is reviewed and approved.
4. Code is merged into the QA branch.
Binary Repository
Artifactory
1. GCloud SDK, GCloud Authentication and build scripts are pulled from Artifactory
2. Successful build deploys the resource configurations to the Kubernetes cluster.
Backend Pod - QA branch
GitHub
The new commit triggers a build on the QA branch in Travis CI
1. The images specified in the deployment are pulled from Artifactory
2. A rolling update updates the underlying resources
Continuous Integration - Single/Multiple Container Promotion

0.0.170
DEV
fe-web: 1.3.3
Environment to Environment Releases
be-users: 2.8.1
be-trx: 1.6.0
be-alerts: 1.2.9
0.0.45
UAT
fe-web: 1.2.2
be-users: 1.0.0
be-trx: 1.2.0
be-alerts: 1.2.9
0.0.3
PRD
fe-web: 1.1.0
be-users: 1.0.0
be-trx: 1.1.0
be-alerts: 1.0.0

This repository has a JSON schema representation of
every pod and all of the underlying containers per
pod. The values to each pod represent the current
version in that environment.
Environments Repository - UAT branch
GitHub
Continuous Integration - Environment to Environment Promotion (QA to UAT)
Web Pod - UAT branch
GitHub
1. Build individually checks out the UAT branch in every
pod defined in the environments.json configuration.
2. Every image in the pod repository UAT branch is
updated to reflect the version in environments.json of
the environments repository for that pod.
3. The changes to each pods deployment.yaml are
committed in the UAT branch of the pod repository.
1. Pull request created in GitHub from QA to UAT.
2. Pull request is approved.
3. Code is merged into UAT with the versions from QA.
Binary Repository
Artifactory
The new commit triggers a build on the UAT branch in Travis CI
Environments Repository Build - UAT branch
Travis CI
1. The images specified in the deployment are pulled from Artifactory.
2. A rolling update updates the underlying resources.
Backend Pod (Inventory Service) - UAT branch
GitHub
Backend Pod (User Service) - UAT branch
GitHub
The new commits trigger builds on every pod UAT
branch in Travis CI.
Backend Pod (Inventory Service) Build - UAT
branch
GitHub
Backend Pod (Inventory Service) Build - UAT
branch
GitHub
1. GCloud SDK,
GCloud
Authentication and
build scripts are
pulled from
Artifactory.
2. Successful build
deploys the resource
configurations to the
Kubernetes cluster.
Web Pod Build - UAT branch
Travis CI

Future Development
● Meta-tagged software releases
● Helm
● Reduction of build steps
● Cached directories in Travis
● Optimized Docker image sizes
● Templated Kubernetes deployments

Questions?