Concourse, Spinnaker, Cloud Foundry, Oh My! Creating Sophisticated Deployment Workflows - Cameron Stewart

© Copyright 2019 Pivotal Software, Inc. All rights Reserved.
Cameron Stewart, Partner Platform Architect
SpringOne Tour Burlington
July 26th
Concourse, Spinnaker, CF, Oh My!
Automate all the things.

Cover w/ Image
Toil
Toil Deﬁned
“So what is toil? Toil is the kind of work
tied to running a production service that
tends to be manual, repetitive,
automatable, tactical, devoid of
enduring value, and that scales linearly
as a service grows.”
Chapter 5 - Eliminating Toil

“... delivering useful software shapes our world…
...[software] is only useful when we deliver it.”
- you’ll see

Getting code into Production is the
most important thing we can do

Everything as Code
➔ You need automation; and you need to be able to manage your infrastructure programmatically -
at every layer
➔ As delivery velocity increases, we need to adopt the practices to deploy applications to other
areas
➔ Infrastructure as Code - writing code that provisions and manages our infrastructure in a
predictable way
➔ Delivery pipelines as ﬁrst class citizens

“We recognize that delivering useful software shapes our world. We recognize that code is the best way to specify precise
action. We recognize that code is only useful when we deliver it.
Delivery is not a detail, it is our job. Now is the time to apply our core skills to our own work. Now is the time to engineer
our delivery. We divide our work between ourselves and computers: humans for decisions, and automation for tasks.”

Continuous Delivery
“Continuous Delivery is about shipping small
batches of software to production constantly,
through automation. Continuous delivery makes the
act of releasing dull and reliable, so organizations
can deliver frequently, at less risk, and get feedback
faster from end users.”

Getting code into is the most
important thing we can do

Continuous Delivery Pipeline
Automated builds
Unit tests
Compliance checks
Service tickets
Performance tests
Security validation
Blue/Green deploys
Canary analysis
A/B testing
Monitoring
Security scans
Chaos engineering
Test-driven dev
Iterative coding/ﬁxing
Frequent integration
Commit code change
Store binaries &
build artifacts

Continuous Delivery Pipeline
Automated builds
Unit tests
Blue/Green deploys
Canary analysis
A/B testing
Monitoring
Security scans
Chaos engineering
Test-driven dev
Iterative coding/ﬁxing
Frequent integration
Commit code change
Store binaries &
build artifacts
Continuous Integration
Continuous
Delivery
Compliance checks
Service tickets
Performance tests
Security validation

The Concourse Pipeline
Flexible integration
of resources
Simple modeling of
components
Pipeline status is
immediately visible
Build components are
expressed as code

Concourse Concepts: Simple Primitives
Resources
Detecting, fetching, creation of
externally versioned “things”
Jobs
Compose resources and tasks
together to do something (run tests,
ship, etc).
Tasks
Run a script in a container with its
dependent inputs
# pipeline.yml
resources:
- name: source-code
type: git
source:
uri: https://github.com/...
branch: master
- name: source-code
type: git
source:
uri: https://github.com/...
branch: master
# pipeline.yml
jobs:
- name: unit
plan:
- get: source-code
trigger: true
- task: unit-tests
file: source-code/ci/unit.yml
# unit.yml
platform: linux
image_resource:
type: docker-image
source:
repository: java
tag: '8'
inputs:
- name: source-code
run:
path: source-code/mvnw
args: [ clean, test ]

Concourse Pipeline Visualization

Platform Automation
Automate the installation and updates
of PCF to stay stable and secure always
● Concourse is an essential part of
your upgrade path, enabling you
to automate small, deﬁned actions
for interacting with OpsManager.
● The resulting perpetual upgrade
machine helps you to maintain
secure, stable PCF instantiations
all the time.
● It also ensures developers always
have access to the latest features
and capabilities.
Application CI/CD
Automate the CI/CD pipeline to deliver
apps/services to PCF fast and safely
● Concourse presents a general
approach to automation that
makes it ideal for CI/CD
workﬂows.
● Think Concourse for automating
continuous integration (CI) and
production deployment readiness
(e.g., automate change tickets,
compliance, security).
● Integrate with CD solutions like
Spinnaker for continuous
delivery/deployment
ApplicationPlatformConcourse
for PCF Use
Cases
Automate Modern Cloud Delivery

Embedded OS
(Windows & Linux)
NSX-T
CPI (15 methods)
v1
v2
v3
...
CVEs
Product Updates
Java | .NET | NodeJS
Pivotal Application
Service (PAS)
Application Code & Frameworks
Buildpacks | Spring Boot | Spring Cloud |
Steeltoe
Elastic | Packaged Software | Spark
Pivotal Container
Service (PKS)
YOU build the containerWE build the container
vSphere
Azure &
Azure StackGoogle CloudAWSOpenstack
Pivotal
Network
“3Rs”
Github
Concourse
Concourse
Pivotal Services
Marketplace
Pivotal and
Partner Products
Continuous
delivery
Public Cloud
Services
Customer
Managed
Services
OpenServiceBrokerAPI
Repair
— CVEs
Repave Rotate
— Credhub

What’s good
for your apps,
is good for
your platform.
“
”

Continuously Delivering Your Platform

Cover w/ Image
(Spring) Cloud Pipelines
➔ Creation of a common deployment pipelines in
Concourse or Jenkins
➔ Propagation of good testing and deployment
practices
➔ Reducing the time required to create pipelines
➔ Project crawler
➔ Implements tests for API contracts using Spring
Cloud Contract
➔ Integrates with ﬂyway Database schema
migrations

Spinnaker Is an OSS Multi-Cloud Delivery Platform
Spinnaker Community
“...the passionate open source
community dedicated to making
deployment pain go away.”
https://www.spinnaker.io/publications/ebook/

Sophisticated Deployment Strategies
https://www.spinnaker.io/concepts/

Application Centric Control Plane
=

Spinnaker microservices
https://www.spinnaker.io/reference/architecture/

Spinnaker
Ecosystem
Cloud Providers
App Engine
Amazon Web Services
Azure
Cloud Foundry
DC/OS
Google Compute Engine
Kubernetes
Openstack
Oracle
CI Systems
Jenkins
Travis CI
Wercker
Concourse
Artifact Support
Docker
Google Cloud Storage
GitHub
HTTP
S3
Artifactory
Monitoring
Datadog
Prometheus
Stackdriver
Atlas
SignalFx
(10+ more on
roadmap)
Notiﬁcations
Email
HipChat
Slack
SMS via Twilio

Stateless: Rather than sharing state, every task
runs in its own container, controlling its own
dependencies.
Ephemeral: Clean and consistent builds.
Minimizes test infrastructure to maintain and gives
you ﬂexibility to scale with the cloud.
Continuous integration: Track versions of
external artifacts used for CI (e.g., Git, S3, docker
image). Build components expressed as code with
simple modeling of modular components.
Flexible workﬂow automation: Extensible for
compliance or other customized tasks in a
pipeline.
Stateful: Maintains awareness of the topography
(multiple clouds, PCF Foundations, etc.) and
keeps record of all pipeline activities for
compliance and audit.
Application inventory: Maintains inventory of
deployed applications, supporting activities like
rollback, canary analysis and identifying security
vulnerabilities.
Sophisticated deployment scenarios: Canary
deploys, red/black or blue/green deploys,
progressive deployments (e.g., by time zone),
multi-cloud/PCF Foundation deployments.
How Does Spinnaker Relate to Concourse?

Interactive Continuous Delivery Pipeline
CI
Production
Arbitrary Jobs

Spinnaker +
Concourse
Together
Bidirectional Integration
● Using Concourse as a trigger
○ Trigger a Spinnaker pipeline stage
○ Pass Concourse trigger info on to
subsequent stages
● Including Concourse as part of a
Spinnaker pipeline
○ Create a stage that watches
Concourse jobs
○ Concourse tells Spinnaker what jobs
to watch
○ When Concourse jobs complete,
Spinnaker stage completes

Using Concourse trigger info in subsequent stages

Using Concourse trigger info in subsequent stages
Ex: use the version to identify
the application binary in a
Deploy stage

1. Concourse job runs
Any Concourse out resource
metadata will be available to a
running Spinnaker pipeline.

2. Spinnaker pipeline is triggered

Stage Conﬁguration
The Concourse resource uses
these to identify whether a
running stage is a match against
the job it is attached to.

2. Concourse check resource identiﬁes the stage.

3. Concourse tells Spinnaker which job to watch.

5. Spinnaker sees the Concourse job is ﬁnished,
and completes the stage.

Virtuous cycle of
automated and iterative
progression, feedback,
and improvement of
software
Continuous Delivery to the Cloud
Monitoring
Securing
Resiliency
Develop
Testing
Experimenting
Compliance
Pivotal Cloud Foundry
Deploy Operate
Planning
Test-Driven Dev
CI
Development Users

Concourse, Spinnaker, Cloud Foundry, Oh My! Creating Sophisticated Deployment Workflows - Cameron Stewart

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Concourse, Spinnaker, Cloud Foundry, Oh My! Creating Sophisticated Deployment Workflows - Cameron Stewart

Semelhante a Concourse, Spinnaker, Cloud Foundry, Oh My! Creating Sophisticated Deployment Workflows - Cameron Stewart (20)

Mais de VMware Tanzu

Mais de VMware Tanzu (20)

Último

Último (20)

Concourse, Spinnaker, Cloud Foundry, Oh My! Creating Sophisticated Deployment Workflows - Cameron Stewart