A Joint Webinar of DBmaestro and Delphix: Manage Databases Like Codebases

1. Thank you for joining us, the webinar will start at:
10:00 Pacific / 12:00 Central / 13:00 East / 18:00 UK Time
Manage Databases like Codebases

2. Before we start
• You will be on mute for the duration of the event
• We are now talking so please type a message
in the Questions box in the Control Panel if you can’t
hear us (please check your speakers and
GoToWebinar audio settings first)
• There will be a Q+A session at the end but please
feel free to type your questions in the Questions
box in the Control Panel in advance
• A recording of the full webinar will be put up online

3. Presenters
Tim O'Brien
• Analyst, CTO
• Gleanster Research
Yaniv Yehuda @yanivyehuda
• Co-Founder & CTO at DBmaestro
• Presenter at world-wide conferences: ODTUG, ilOUG
Kyle Hailey @kylehhailey
• Technical Evangelist at Delphix
• Oracle ACE, member of the OakTable Network

4. About Delphix
• Founded in 2008, launched in 2010
• CEO Jedidiah Yueh (founder of Avamar: >$1B revenue))
• Based in Silicon Valley, Global Operations
• 10% of Fortune 500

5. About DBmaestro
• Founded in 2008, launched in 2010
• Founded by Yariv Tabac and Yaniv Yehuda
• Headquartered in Israel, Global Operations

6. Manage Databases like Codebases
Kyle Hailey & Yaniv Yehuda

7. The Business Need
Copyright@2010, Gartner RAS Core Research
80%
of unplanned
downtime is
due to Change
50%
More than
of this, half is
due to human
errors
40% of changes FAIL Copyright@2008, Juniper Networks, Inc.

8. Code, Databases,and
Agility in 2014
May20,2014
Tim O’Brien
Analyst, CTO
Gleanster Research

9. Industry
Trends
Code in 2014
 Distributed &
Asynchronous New Normal
 Self-service Critical to
Productivity
 Increasing Variety (from
Javascript, Java, .NET, Ruby,
Python + everything)
Everything is Automated

10. Industry
Trends
Databases in 2014
 More than just Relational
 Increasing Variety
(Oracle, Mongo)
 Management? Not as
Mature as Code
Oddity
 Big Data marginally more
manageable than
traditional RDBMS
 “We’re still waiting for
the database…”

11. Code is King
Modern Enterprise
 Fully distributed development
 Continuous Deployment Pipelines
 Everything Automated! -
DevOps, PaaS, IaaS
“Every Company is a Software
Company”
How did we get here?

12. Scale
Development
Open Source @ Scale
 Google Chrome, Mozilla Firefox
 Android, Apache httpd
 Linux*, FreeBSD
Characteristics
 Rapid Iterations, Frequent Releases
 Numerous Experiments (or Branches)
 Thousands of developers
 OSS @ Scale established best practices
now in use.

13. Scale
Development
Organizations @ Scale
 Facebook,Wal-
Mart, eBay, Salesforce, Intuit
 Goldman Sachs, Apple, Amazon,Yahoo!
 Google, SSA, Ford,Toyota
Characteristics
 Rapid Iterations, Frequent Releases
 Numerous Experiments (or Branches)
 Thousands of developers, ManyTeams

14. @Scale
Everything is
Continuous
Continuous integration and
deployment
Modern set ofTools Integrated with:
 Source Control – Perforce, Clearcase, Git
 Infrastructure-as-a-Service – Openstack
 Platform-as-a-Service – AWS, Heroku
 Databases? - Still an Integration Nightmare

15. Case in Point
 Example Fortune 500 Relaunch
 Re-architecture: More distributed, Move to
a PaaS
 Scope: Everything, 2Year Migration Project
 Size: 1500 Developers, Several
Continents, $100M+
 Results?
 Code: Distributed Projects, Continuous
Deployment
 Infrastructure: Self-service, On-
demand,Agile
 Database: Delays, Manual Builds, Cost
Overruns

16. RootCause
Analysis
 Code is Agile
 Developers move FAST.
 Branches are fungible, disposable, personal.
 Deployments managed with DevOps tools
(immediately)
 Traditional Databases are Not
 Costly to Setup
 Often require Dedicated Hardware
 Little automation. No Change Management
“The rest of the department is
Virtual but our Databases are
stuck in 1998.”

17. Worst
Practices
(we’reall used to)
 Lack of Database Change Management:
 Shared Developer Databases Results in Contention
 Costly, Slow Environment Provisioning
 Testing and Staging Not Synchronized with
Production
 Drag on Productivity and Efficiency
 Opportunity lost
 “We’ve accepted that the databases causes
delays”

18. What We’ve Seen
1. QA: Expensive, Slow
2. Development: Bottlenecks & bugs
3. Provisioning: Delays

19. 1. QA: Expensive, Slow : Long Build times
Build Time
96% of QA time was building environment
$.04/$1.00 actual testing vs. setup
Build
Build Time
QA
Test
QA
Test
Build

20. 1. QA: Expensive, Slow: bugs found late
Build QA Env QA Build QA Env QA
Sprint 1 Sprint 2 Sprint 3
Bug CodeX
Delay in Fixing the bug
Cost
To
Correct
Software Engineering Economics – Barry Boehm (1981)
Build QA Env QA

21. 2. Development: Bottlenecks & bugs: Bottlenecks
Frustration Waiting

22. 2. Development: Bottlenecks & bugs: subsets cause
bugs

23. 2. Development: Bottlenecks & bugs: subsets cause
bugs

24. 3. Provisioning Delays : weeks to months
Management
DBA
System Admin
Storage Admin
Developers
Submit
Request
Disk
Capacity?
Approve
Request $$
(2 Weeks)
Approve
Request $$
(1 Week)
Request
Additional
Storage?
Provision
Capacity
File System
Configured?
Configure LUNS &
Build File System
Coordinate
Replication w/
Infrastructure
Re-
Parameterize &
Configure DB
Mount
Recovery DB
to Specific PIT
Begin
Work
Approve
Request $$
(2 Weeks)
(3 Days)
(3 Days)
(2 Days)
(3 Days)
(3 Days)
…….1-2 Weeks of Approvals, Delays, and Provisioning……
24
Developer Asks for DB Get Access
Manager approves
DBA Request
system
Setup DB
System
Admin
Request
storage
Setup
machine
Storage
Admin
Allocate
storage
(take snapshot)

25. 3. Provisioning Delays : culture of no

26. What We’ve Seen
1. QA: Higher costs more code re-work
2. Development: Bottlenecks & bugs
3. Provisioning: Delays

27. Three Physical Copies Three Virtual Copies

28. Install Delphix on Commodity Intel Hardware
Intel hardware

29. Allocate Any Storage to Delphix
Allocate Storage
Any type

30. One time backup of source database
Database
Production
Instance
File systemFile system

31. DxFS (Delphix) Compress Data
Database
Production
Instance
Data is
compressed
typically 1/3
size
File system

32. Incremental forever change collection
Database
Production
Instance
File system
Changes
• Collected incrementally forever
• Old data purged
Time Window
File system

33. Typical Architecture
Production
Instance
Development
Instance
QA
Instance
UAT
Instance
Database
File systemFile system
Database
File systemFile system
Database
File systemFile system
Database
File systemFile system

34. With Delphix
Database
File system
Production
Instance
Database
Development
Instance
Database
QA
Instance
Database
UAT
Instance

35. Parallel environments
Instance
Instance
Instance
Instance
Source

37. What We’ve Seen With Delphix
1. QA: Low cost, fast bug detection
2. Development: Parallelized, less bugs
3. Provisioning: Fast, Culture of Yes

38. 1. QA Efficient : Lower cost
B
u
i
l
d
T
i
m
e
QA Test
1% of QA time was building environment
$.99/$1.00 actual testing vs. setup
Build Time
QA Test
Build

39. 1. QA Fast : bugs found fast and fixed
Sprint 1 Sprint 2 Sprint 3
Bug CodeX
QA QA
Build QA
Env
Q
A
Build QA
Env
Q
A
Sprint 1 Sprint 2 Sprint 3
Bug
Cod
e
X

40. 2. Development : Parallelize
gif by Steve Karam

41. 2. Development: Eliminate bugs

42. 3. Provisioning: Fast, Efficient, Self Service, Culture of
Yes!

43. What We’ve Seen With Delphix
1. QA: Low cost, fast bug detection
2. Development: Parallelized, less bugs
3. Provisioning: Fast, Culture of Yes
But …
How do we manage database changes as code
changes and automate deployment of changes?

44. Dealing with Risk
 Smaller and more focused changes are easier to manage (Agile…)
 Automation of repeating tasks lowers risk of (human) error
 Development and Operations should work in synergy (DevOps)

45. 45
• More rapid changes
• Reacting quickly to market needs
• Getting ahead of competition
• Fewer changes backed out
• Better collaboration
• Fewer defects
• Ultimately better service
• Happy customers
• Profitability
Why Continuous Delivery?

46. 46
• Team and process
• Version everything
• Automate your tests
• Fix it, properly (no out of process changes!)
• Automate your deployments
Database Continuous Delivery?

47. 47
A quick poll

48. 48
• The database holds your essential information
• Changes can impact the entire system
• Need to be synchronized with other changes
• Often overlooked
Database is a Key Component

49. 49
• There is more to a database than SQL scripts
– Schema structure
– Code
– Content and meta-content
– Internal dependencies
– …
• Ensure that changes are not made without
authorization
• Ensure no out-of-process changes
Reaching Inside the Database

50. 50
• Old adage but true
• The database is often neglected and therefore can
become the weakest link
• Essential from a compliance point of view
• Should be the strongest link
The Weakest Link In a Chain

51. 51
Challenges…
Un-coordinated Process
Silos in development
Deployment delays…
Out-of-Process Changes
Errors in production
Lack of confidence in automation
Code overrides
Different methodologies
Lack of history
Missing changes
wrong versions

52. 52
Two isolated Processes
Version Control Process Development Process
Check-Out
Script
Modify Script
Get updated
Script from DB
Check-In
Script
Compile
Script
in DB
Debug Script
in DB
?
?
?
?
A
A’

53. 53
X
1.11.1.11.11.21.31.41.51.61.7
Build Once Deploy Many
Int QA Stage Prod
Database Deploy Script
Environment
Re-Base (due to defects)
Dev
Dev
Dev
Model
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.11.11.41.7
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
Out of Process
Change
X
X
X
X
X
? 1.1.1
X

54. 54
Dealing with challenges…
Coordinated ProcessTraceability
Start in the Beginning
No Out-of-Process Changes
Impact Analysis
Automation
Task Based Development
Well Defined Processes

55. What is DBmaestro TeamWork
• Database Enforced Change Management
+ Database version control
+ Plugs into the ALM (change request, tickets & work items)
+ Database merge & change impact analysis
• DevOps Solution for databases
+ Baseline aware deployment automation, rollback &
recovery
+ Plugs into release management & Continuous
Delivery

56. How?
• Database version control
– Enforced Check Out/In
– Labels
– Rollback/Undo
– Audit trail reports
• Database impact analysis
– Utilizes version control repository information
– 3-way analysis
• Database deployment automation
– API
– Baselines
– Conflict resolution
– Customized business logic

57. 57
Version Control - One Enforced Process

58. 58
1.11.21.31.41.51.61.7
Build & Deploy On Demand
*
Int QA Stage Prod
Database Deploy Script
Environment* Execute the same script
being executed at the
Stage environment
Re-Base (due to defects)
Dev
Dev
Dev
Model
1.1 1.2
1.2 1.3
1.3 1.4
1.4 1.5
1.5 1.6
1.6 1.7
1.1 1.4
1.4 1.7
1.1.1 1.7
1.1 1.1 1.11.41.7
File Based
Version Control
Out of Process
Change
1.1.11.7 1.1.11.7

59. Safety Net For Deployment Automation
Source vs.
Target
Action
= No Action
≠
?
Source vs.
Baseline
Target vs.
Baseline
Action
= = No Action
≠ = Deploy Changes
= ≠ Protect Target
≠ ≠ Merge Changes
You do not have all
of the information
With Baselines and 3 way
analysis the unknown is
now known
Simple Compare & Sync Baseline aware Deployment
An index exists in Target (Production) but not in source (QA).
What should we do? Drop the index or not?

60. Benefits - Development
• Database change repository
• Follow SCM best practices (Check-Out/Check-In)
• All changes are documented
• Manage who can do what, where, when & why

61. Benefits - Operations
• Integrated deployment engine
• Business level audit
• Roles & responsibilities enforcement

62. Benefits - Management
• Complete visibility into changes in progress
• Management reports, Audit reports
• No silos

63. +

64. Time Window
Instance
Instance
Instance
Virtual DatabaseDev1
Dev2
Trunk
Virtual Database
Virtual Database
Developer 1 modify
Developer 2
modify
DB
VC

65. Dev2
Dev1Merge to dev1
Trunk
DB
VC
Fork
Fork
Fork
Fork

66. Safe and Efficient Work Process
• Clone relevant number of virtual copies of the Trunk
– Dev Team 1-n, Developer a-z, Hot fix environment, Etc…
– Work in parallel, save time
• Rely on enforced change process
– Know exactly who did what, when and why
• Deployment automation
– Save time, mitigate risk
– Continuous integration, Continuous Delivery
• Deal with conflicts & merge them into the Trunk
• Test before deploy to production
– Pre-prod clone

67. Q&A
Kyle Hailey @kylehhailey
Delphix: delphix.com
Yaniv Yehuda @yanivyehuda
DBmaestro: dbmaestro.com
Tim O'Brien
Gleanster: gleanster.com

68. Thanks!
Kyle Hailey @kylehhailey
Delphix: delphix.com
Yaniv Yehuda @yanivyehuda
DBmaestro: dbmaestro.com
Tim O'Brien
Gleanster: gleanster.com