Continuous Delivery for Databases

1. Continuous Delivery for Databases
Presented by: James Smith
Co-Founder, DevOpsGuys

2. ABOUT ME
Co-founder @ DevOpsGuys
Two decades building & delivering enterprise web systems
Helped many, many companies implement Continuous Delivery
practices
Found High Quality Belgian Beer in late nineties!
Who is this guy?

3. WHY?
Why is this important?

4. WHY?
Why is this important?

5. BUSINESS HAS CHANGED

6. SPEED MATTERS

7. 7

8. DEPLOYMENT RATES
8
300 Deployments / Year
50-60 Deployments / Day
10+ Deployments / Day
Every 11.6 seconds

9. HIGH PERFORMERS
HIGH PERFORMING IT
ORGANISATIONS DEPLOY 30
TIMES MORE FREQUENTLY, WITH
50% FEWER FAILURES WITH
8000X FASTER LEAD TIMES THAN
THEIR PEERS.
9

10. HIGH PERFORMERS
THEY ARE ALSO TWO TIMES MORE LIKELY TO
EXCEED PROFITABILITY, MARKET SHARE &
PRODUCTIVITY GOALS
THEY EXPERIENCE 50% HIGHER MARKET
CAPITALIZATION GROWTH OVER 3 YEARS
10

11. HOW?
How have things changed?
11

12. 2010
2012
2001 (1998)
2001 (and earlier)
DEV HAS CHANGED

13. WE KNOW AGILE WORKS

14. REGULAR, RAPID, RELIABLE AND
CONTROLLED DELIVERY OF
WORKING SOFTWARE SYSTEMS
INTO PRODUCTION
Matthew Skelton

15. DEVOPS CALMS MODEL
Culture
 Hearts & Minds, Embrace Change
Automation
 Of all the things; testing, deployment, infrastructure
Lean
 Small batch sizes, value for end-users
Measurement
 Of all the things; show the improvement
Sharing
 Open, transparent collaboration
WWW.DEVOPSGUYS.COM - @DEVOPSGUYS 15

16. Plan Code Build Test Release Deploy Operate
CONTINUOUS ?
16
Continuous Integration
Continuous Delivery
Continuous Deployment
DevOps

17. Nhan Ngo, a QA engineer at Spotify

18. CONTINUOUS DELIVERY
1. Build Management & CI
2. Environments & Deployment
3. Release Management & Compliance
4. Testing
5. Data Management
18

20. Your systems are under pinned by data and in most organisations
data is the hardest and most expensive item to manage.
Data is needed by developers and testers to ensure that software
products are high quality. Its needed by support teams to
troubleshoot and resolve issue effectively.
20

21. BUT DATABASE CHANGE IS
SCARY
21
Photo Credit: http://cache.lego.com/r/www/r/movie/-
/media/franchises/the%20lego%20movie/explore/downloads/wallpapers/lego_w
ps_1600_emmet.jpg?l.r=-1410608028

22. NO, THIS IS SCARY
Photo Credit: His mum

23. SO WE MINIMISE RISK
Photo Credit: https://ideas.lego.com/projects/62456

24. WE DEFEND THE THREAT
Photo Credit: "Rumrunner" via Compfight cc

25. WE STOP CHANGE
25
Photo Credit: s.kosoris via Compfight cc

26. CUSTOMERS GET SAD
26
Photo Credit: Kalexanderson via Compfight cc

27. THE BUSINESS GETS ANGRY
27
Photo Credit: powerpig builds via Compfight cc

28. AND IT GET THE BLAME!
28
Photo Credit: bobsfever via Compfight cc

29. Dev are shipping application changes frequently
Ops are creating servers - automatically
29

30. WHY IS THERE SO MUCH LEGO
IN YOUR SLIDE DECK?
Perfectly normal question

31. LEGO = DEVOPS
Rapid proto-typing & experimentation
Building blocks no right or wrong way
Promotes collaboration
Strong cultural appeal
Small batch sizes
Visibly measureable

 Manufacturing
 Zenon
 Mindstorms
Danish phrase leg godt, which means "play well".

32. WHY?
Why is database automation so hard?

33. != CTRL C, CTRL V
Database deployment is not copying and replacing.
It is the transformation from a previous version to the next
version while preserving data integrity.
Deploying database change is hard
Deploying database change frequently is even harder

34. WHAT?
What are the common challenges?

35. #1: VERSION CONTROL
Photo Credit: ntr23 via Compfight cc

36. VERSION CONTROL
 Database changes not under VCS
 Worse
Communication of change
 t

37. #2: AGAINST THE FLOW
37
Photo Credit: slang589 via Compfight cc
Photo Credit: slang589 via Compfight cc

38. AGAINST THE FLOW
DEV PRODUCTION
38
DEV PRODUCTION
CODE
DATA

39. #3: COMPLEXITY
39
Photo Credit: L@go via Compfight cc

40. COMPLEXITY
Dependence
Centralisation

41. #4: DATA VOLUME
Photo Credit: http://www.calgaryherald.com/news/calgary/Gallery+LEGO+KidsFest/9848406/story.html

42. DATA VOLUME
Transit time
Historical data no archiving
One of the biggest impacts on Cycle time

43. #5: CONWAY'S LAW
Photo Credit: kirk_arts via Compfight cc

44. CONWAY'S LAW
"Any organization that designs a system (defined more broadly
here than just information systems) will inevitably produce a
design whose structure is a copy of the organization's
communication structure
- Conway, 1968
44

45. CONWAY'S LAW
Is (was) a database really needed? 3 Tiers anyone?
Centralised vs Decentralised?
Formalised [change] control

46. HOW?
So what are some of the solutions?

47. #1: GET UNDER CONTROL
Photo Credit: RHiNO NEAL via Compfight cc

48. VERSION CONTROL
 it should be in VCS!
 it should be in VCS!
 it should be in VCS!
Schema & Static/Reference Data
Reverse engineer existing schema & reference data

49. #2: INTEGRATE
CONTINUOUSLY
49Photo Credit: ....Tim via Compfight cc

50. CONTINUOUS INTEGRATION
 Automate your build steps (Full vs Incremental)
Build a library of manual tests
Automate your tests
50

51. #3: AUTOMATE TESTING
51
Photo Credit: s3aphotography via Compfight cc

52. TEST, TEST, TEST
Select the right tests for each stage;
Unit testing
Integration Testing
Deployment Validation
Behaviour Validation
Determine the right data for testing
Do you need it all?
52

53. #4: AUTOMATION
53Photo Credit: pasukaru76 via Compfight cc

54. AUTOMATE ALL THE THINGS
Testing
Deployment
Back-up and more importantly restore
Archiving
Rollback
54

55. #5: MICROSERVICES
55

56. RE-ARCHITECT
Small is good
Split data along sensible partitions
Polyglot persistence
56

57. POLYGLOT PERSISTENCE
57

58. #6: CHANGE YOUR WAYS
58Photo Credit: Stéfan via Compfight cc

59. Technology is only half of the story

Management must think of operations as part of development
Deployment is part of development
Data retention is part of development
Fail faster, but fail safely
59

60. TOOLING
Can tools help?
60

61. SOURCE CONTROL
Source control is the foundation of Continuous Integration
Plus many, many more

62. CONTINUOUS INTEGRATION
Continuous Integration is the common orchestration point
62

63. DEPLOYMENT
Repeatable and consistent deployments every time.
63

64. REDGATE DLM (SQL SERVER)
64

65. MONITORING
Continuous delivery elevates the need for monitoring in
production

66. THE TRUTH
Continuous Delivery relies on having 2 basic things;
1. Version Control
2. Automation

67. Customers see results and new features more quickly.
Shorter feedback cycles increases our ability to learn.
Improve the whole system.
Reduce firefighting.
Everyone wins!
67

68. EVERYBODY WINS!
68Photo Credit: Kalexanderson via Compfight cc

69. 69Photo Credit: Walter Benson via Compfight cc