3. Quem Somos
Miguel Alho, Software Engineer @ Faro
@MytyMyky
https://pt.linkedin.com/in/miguelalho-982331
alho@miguelalho.com
http://www.miguelalho.pt/
4. Chapter 1: Introduction
1. The “DevOps” context
2. The problem that we want to fix
3. The database challenge
4. Infrastructure
5. Collaboration & Culture
6. Continuous Integration
7. Continuous Delivery
5. Chapter 2: Application
1. Microservice repository creation
2. Creating the first functionality with TDD
3. Implement a build job / CI pipeline
6. Chapter 3: Infrastructure
1. Physical machines
2. Virtual machines
3. Platform as a Service
4. Serverless
5. Infrastructure as Code
6. Provision and deployment for our microservice
7. Chapter 4: Database
1. The special case of the database
2. Database automation
3. Migrations vs State
4. Flyway
5. Database integration into our microservice
8. Chapter 5: Managing Changes
1. Business requirements changes
2. How interaction is important to determine how to move forward
3. Versioning strategies
4. Deployment strategies
5. Implementing changes in our Microservice
12. Stakeholders
Sponsors / Customers
Product Owner / Project Manager
Coaches and Scrum Masters
Business Stakeholders
Architects, Leads and Developers
Quality Assurance Engineers
Release Engineers
DBAs
IT Operations
Development Stakeholders
User
13. Business stakeholders want...
Working software
Software that fix the customer's problems
Software that add value
New features and fast
Incremental features
Frequent changes (Continuous Change)
Business development and continuity
Sponsors / Costumers
Product Owner / Project Manager
Coaches and Scrum Masters
Business Stakeholders
Sponsors / Customers
Product Owner / Project Manager
Coaches and Scrum Masters
Business Stakeholders
14. Development stakeholders want...
Development Stakeholders
Develop quality solutions
Secure and stable solutions
Avoid bugs and midnight calls
Few obstacles
Architects, Leads and Developers
Quality Assurance Engineers
Release Engineers
DBAs
IT Operations
15. There is a problem
that we need to fix
Chapter 1
16. Dichotomy of interests
Traditionally there is collision of interests.
Devops intends to break this collision with communication, empathy and
collaboration improvement, along with useful technical practices, principally
automation
New features
Corrections and improvements
Optimizations
Stability
Security
Troubleshooting
Operations Devs
17. Delivery of new features quickly
Devs must be able to make changes to software quickly to integrate new features
Software should be developed in order to integrate changes, instead of simply
passing the tests
Requires:
● Use of adequate design patterns
● Automation of the build process
● Testing in different environments/levels
● Fast functionality validation
● Regression prevention
● Quality measurement
18. Software delivery consistently
Putting software in production should not be a headache
Requires:
● Continuous Integration
● Continuous Delivery
● Process automation – consistency guarantee
● Visibility – quality and metrics
19. Scalability
Rapidly answer to new needs for delivery as well as users load requirements
Requires:
● Provisioning automation
● Deployment automation
● Instance generation simplification
● Monitoring
21. Database as bottleneck
● Databases traditionally are out of pace with application development
● Easy target to blame
● Lack of traceability of databases changes (changes history)
● Manual database processes prevent CI and CD
22. Database as bottleneck
Requires:
● Automation
● Source Control database changes (1st step for db deployment pipeline)
● Synchronization between application and database
24. Infrastructure
How can you have the required consistency without including infrastructure?
If you practice (development) and play (production) in different fields, why do you
expect the same behavior/performance?
34. Knight(Mare)
● Knight Capital Group
● 3.3 billion trades daily
● $21.5 billion traded daily
● $365M cash & equivalents
-----
● NYSE launches Retail Liquidity Program on August 1, 2012, an automated
routing system for equity orders
● Group updates software to comply and repurpose 8-year old feature flag
35. Knight(mare)
“During the deployment of the new code, however, one of Knight’s technicians
did not copy the new code to one of the eight SMARs computer servers.”
SEC Filing | Release No. 70694 | October 16, 2013 ( http://tinyurl.com/Devops-Knightmare )
36. Knight(mare)
● 9:30 AM - Market Opens
● 212 small retail “parent-orders”
● 7 servers processing “child-orders” correctly.
● 8th server using the old feature flag code failed to recognize parent-orders
were fulfilled.
● 8th server sent cumulative child.-orders in rapid succession.
37. Knight(mare)
● No automated fail-safe
● No procedures for how to react
● Knight uninstalled the correct SMARS code from the seven servers where
it had been deployed correctly
● 9:30 AM to 10:15
● 4 million executions in 154 stocks
● 357 million shares
40. Continuous Integration
Increases your opportunities feedback (fast) feedback
Build software at every change
Features:
A connection to a version control repository
A build script
A feedback mechanism
A process for integrating the source code changes
41. Continuous Integration
To implement it, we need:
● Source Control - Git, Hg, Svn, TFS, ...
● Build System
○ Scripting - Cake, Psake, FAKE, Gulp …
○ Compiler and tools - MsBuild, Node/npm, GitVersion, …
○ Build Server - VSTS, TeamCity, AppVeyor, CircleCi
● Testing Framework - nUnit, xUnit, Fixie, tSQLt, ...
● Packaging mechanism - Nuget, Docker container, zip file…
45. Here’s what we are going to do:
● Create a project in VSTS
● Create a repository for a Microservice, with in memory data
● Implement the microservice using TDD
● Create a basic build job for our microservice in VSTS
● Se this whole setup run
48. Virtual Machines
In the cloud
Upfront cost
First step to IaC
Requires configuration management
49. Platform as a Service (PaaS)
Very flexible approach
Costs grow with the application
Fast and simple development
Bring infrastructure/operations closer to the
development
51. Infrastructure as Code (IaC)
Thank you Virtualization!!
Managing and provisioning infrastructure
Is not simply writing scripts
Allows to apply proven software development practices
Version control, testing, small deployments
56. Database challenges
Databases are out of pace with application development
Lack of traceability of database changes (changes history)
Manual databases processes prevent the CI and CD utilization
Lack of testability
Database setup time for new environments
Bugs in production
59. Why database automation?
Enable control over database development
Increase speed of response to change
Keep a versioned “history” of database states and changes
Reliability of the release process
Increase release frequency though repeatability of processes
Reduce time spent fixing bugs - automated tests
Remove/reduce human intervention in the release process
61. 1st Step: Source Control
First step in your database deployment pipeline
Traceability through change history
SQL as documentation
Shared code-base and shared process
Enforceable standards to reduce conflicts
63. Migrations vs State
State based solutions
Script represents the current database state
Your source of truth is how the database should be
Migrations based solutions
Script represents a migration
Migration represents how to transition to the next database version
Your source of truth is how the database should change
64. Flyway
Flyway – “Database Migrations made Easy”
http://flywaydb.org/
Open source database migration tool
Flyway command-line enabled
Simplicity: easy to setup, no need to install
Zero dependencies (java + jdbc)
Scripts are written in SQL
65. Flyway: How it works
Flyway commands:
migrate, clean, info, validate, baseline, repair
Metadata table:
Used to track the state of the database
If the database is empty, flyway won’t find it and will create it
Flyway scans the migrations directory and check migrations against the metadata table
Migrations are sorted based on their version number and applied in order
66. Version Control System
Two components
Database scripts repo (git)
Flyway - database migrations
Rule 1: Script version (timestamp)
Rule 2: Operation type
Rule 3: Object type
Rule 4: Object name Example: V20160728.1100__Create_TB_MyTable.sql
67. Time to add a database to our service
● Provision development Database
● Add test project in Code
● Implement database features in our microservice
● Test
● Add steps in our build system to support this
● Deploy changes
69. Change is unavoidable
Businesses change naturally
● competitive advantage requires enhancements
● products need to move towards customer’s real needs
● social and economic context affects how business is done
Software NEEDS to change to enable this!
70. Change is not easy in software
Ideally, changes contained in one component should NOT break all the others!
Unfortunately, this is not always possible
● more than one app consumes a service
● more than one app consumes a database
● binary data contracts (dlls) need to be updated everywhere
● environment variables and changes affect program’s expected behaviour
● lack of tests brings uncertainty to change outcome
Many of these are anti-patterns!
71. Collaboration is key
● Need for change needs to be communicated
● Risk of change needs to be analysed
● Effects need to be listed to prepare for possible problems and avoid fire-
fighting
● Teams need to announce how changes will impact existing processes and
work.
● Teams need to determine if the change is done in a single step or in multiple
iterations.
72. Versioning strategies
● Versioning allows team to tag artifacts to a specific state
● Collaborating components can assert their validity based on versions
○ component A requires B version X or higher
● Semantic versioning gives meaning to number
○ prefer SemVer
● Automate versioning as much as possible
○ tooling and commit messages can be enough to indicate the type of change
● Important part of a CI pipeline - end result of a build is a specific version of an
76. Versioning is not easy
● But it is extremely important
● Semantics matter
○ minors and patches mean clients shouldn’t have to change anything, only update
○ Majors are breaking so clients need to take some maintenance and realignment action
● It’s also automatic documentation
77. Deployment strategies
You choose based on what your context is (traffic volume, working hours, SLAs)
● Single Server or service
○ Stop service, upgrade, restart
● Multiple Servers
○ Stop service, upgrade, restart
○ Blue-Green deployment
○ Canary Deployment
Whatever the scenario, automation is key
78. Deployments with DBs make things harder
● Database’s data is mutable. It’s super hard to rollback!
○ How do you rollback changes that delete a column of data?
● Application is typically immutable. It’s way easier to rollback
○ Just need the database schema to be compatible
This means that we typically need to make changes in multiple steps to avoid
breaking the service. Schema should valid for backward compatible to at least 1
version of the application. Marking things as deprecated can / should make a
difference.
81. Why DevOps
Developing software is not enough, you have to deliver it
Communication framework for change management
You can not stop change, but you can control it
Perspectives
Need for speed (time-to- market) (management people)
Need for control (error control) (operations people)
82. DevOps: contract for collaboration and communication
Contract - change communication management tool
Change description (Source Control)
Change validation (Continuous Integration)
Change implementation (Continuous Delivery)
Applications, databases, infrastructure
83. DevOps advices
Operations requirements in Backlog
Operations engineers in Product Team
Shared goals Shared responsibilities
Devs monitor apps
Devs deploy apps
