How does data management fit into agile development? How can data professionals take an agile approach to data management? What mindset do data professionals need to succeed in an agile world?

1. Disciplined
Agile
Data
Management

2. Scott Ambler
• Helps enterprise-­class organizations around the
world to improve their processes and organization
structures
• Thought leader of:
– Agile Modeling (AM) method
– Agile Data (AD) method
• Co-­creator of the Disciplined Agile (DA) framework
• Author of 20+ books
• scott [at] scottambler.com
© Disciplined Agile Consortium 2
@scottwambler

3. © Disciplined Agile Consortium
3

4. © Disciplined Agile Consortium 4
Agenda
• The Disciplined Agile Framework
• The Agile Data Mindset
• Disciplined Agile Data Management
• Parting Thoughts

5. Disciplined
Agile Enterprise
Disciplined
Agile IT
Disciplined
DevOps
DAD
The Disciplined Agile Framework
© Disciplined Agile Consortium
http://www.disciplinedagiledelivery.com/the-­agile-­tractor-­engine-­analogy/
5

6. The Disciplined Agile Framework
© Disciplined Agile Consortium 6

7. An Agile Mindset
for Data Management
© Disciplined Agile Consortium 7

8. Evolution over
Definition
© Disciplined Agile Consortium 8

9. Holistic Organization Over Data Management
© Disciplined Agile Consortium 9

10. Collaboration over Documentation
© Disciplined Agile Consortium 10

11. Sufficiency over Perfection
© Disciplined Agile Consortium 11
What we want
Where
Disciplined
Agilists
initially aim
Where
traditionalists
typically aim

12. Cross Functional Over Specialized Staff
© Disciplined Agile Consortium 12
Specialized Staff
Cross-­Functional Staff

13. Automated Over Manual Processes
© Disciplined Agile Consortium 13
Data Automation opportunities:
– Automated regression testing
– Data schema analysis
– Continuous database integration
– Continuous database deployment
– Operational database monitoring with
real-­time dashboards

14. Disciplined Agile Data Management
© Disciplined Agile Consortium 14

15. The Data
Management
Process Blade
© Disciplined Agile Consortium 15

16. Comparing Data Quality Strategies
© Disciplined Agile Consortium 16
Long-­term payback,
difficult to keep updated
Transform logic little more than
a band-­aid over source problems
Ensures that future changes don’t
“break” previous implementations
Detailed executable specification
and validation
Reduces feedback cycle, thereby
minimizing cost of fixing problems
Safely address production problems

17. Evolve Data Quality Assets
• Keep this as light-­weight and collaborative as possible
• Test data management can be challenging:
– Privacy issues surrounding production data
– Some test data will need to be engineered
– Configuration management of test data
– Continuous database integration requires automated restoration of test
data
© Disciplined Agile Consortium 17

18. Refactoring Legacy Data Sources
© Disciplined Agile Consortium 18
Database refactoring
immediately addresses legacy
quality problems BUT requires
long-­term administration to
remove implementation
scaffolding

19. Data Security
• Security addresses a range of issues:
– Data security
– Facilities security
– Staff vetting
– Staff protection
– And more
• Implication: Data management should collaborate closely with
Security professionals
© Disciplined Agile Consortium 19

20. Specifying Data Structures
• Modeling is effective at thinking through and communicating high-­
level ideas
– Fundamental lean concept is that models of complex things should be
very simple in nature, not detailed
• Tests, when automated and better yet written before the
implementation that they validate, are very effective as detailed
specifications
© Disciplined Agile Consortium 20

21. Agile Data Governance
• Effective governance focuses on motivation and enablement, not on
policing the creation of artifacts
• Data governance is one of many aspects of IT Governance
• Agile strategies:
– Collaborative support of teams and stakeholders
– Light-­weight guidance
– Automated monitoring(tests, schema analysis)
© Disciplined Agile Consortium 21

22. Agile Data Quality Strategies
© Disciplined Agile Consortium 22

23. © Disciplined Agile Consortium 23

24. Shifting Your Data Management Mindset
© Disciplined Agile Consortium 24

25. How to Adopt Agile Data Management Strategies
1. Expect (a lot) better
2. Surface your challenges
3. Invest in your staff
4. Hire agile coaches with deep experience in both agile and data
management
5. Invest in automation infrastructure
– Question your existing “data quality tools”
– Data is a critical component of your CI/CD pipeline
– DB testing and refactoring tools are proven in practice
© Disciplined Agile Consortium 25

26. scott [at] scottambler.com
Twitter: @scottwambler
DisciplinedAgileConsortium.org
DisciplinedAgileDelivery.com
Thank You!
© Disciplined Agile Consortium 26

27. Would You Like This Presented
to Your Organization?
Contact us at ScottAmbler.com
© Disciplined Agile Consortium 27

28. Scott Ambler + Associates is the thought leader behind the Disciplined
Agile Delivery (DAD) framework and its application. We are a boutique
IT management consulting firm that advises organizations to be more
effective applying disciplined agile and lean processes within the
context of your business.
Our website is ScottAmbler.com
We can help
© Disciplined Agile Consortium 28

29. Additional
Slides
© Disciplined Agile Consortium 29

30. Values for Agile Data Management
• Evolution over definition
• Holistic organization over data management
• Sufficiency over perfection
• Collaboration over documentation
• Cross-­functional staff over specialized staff
• Automation over manual processes
© Disciplined Agile Consortium 30