My slide deck from SCALE9x / DevOpsDay in LA. I blew the presentation, but hopefully the content had some value.

1. Quality Assurance
in a
DevOps World
Enabling Trust
Kit Plummer
http://kitplummer.github.com
kplummer@maestrodev.com MaestroDev
Tuesday, February 22, 2011 1

2. DevOps is an ecosystem of...
•People (relationships)
• Processes (activities)
• Product(s)
MaestroDev
Tuesday, February 22, 2011 2

3. Ecosystem…
• Applied on top of columnar
organizations:
– Development/SW eng./design/etc.
– Quality assurance/testing/delivery
– Product eng./operations/customers
MaestroDev
Tuesday, February 22, 2011 3

4. Current DevOps thinking is...
• aligned with the Agile Manifesto,
• and prescribes itself as
a Cultural (organizational) and
Professional (personal) movement
.
MaestroDev
Tuesday, February 22, 2011 4

5. MaestroDev
Tuesday, February 22, 2011 5

6. MaestroDev
Tuesday, February 22, 2011 6

7. MaestroDev
Tuesday, February 22, 2011 7

8. MaestroDev
Tuesday, February 22, 2011 8

9. Trust is the cornerstone...
• Patrick Debois' math - based on
Covey's "4 Cores":
• trust = ƒ(character,competence)
• character = ƒ(integrity, intent)
• competence = ƒ(capabilities,
results)
MaestroDev
Tuesday, February 22, 2011 9

10. Trust is a QA deliverable...
• Results are the indicator
• Testing is the apparatus
• Proof of quality is the target
MaestroDev
Tuesday, February 22, 2011 10

11. MaestroDev
Tuesday, February 22, 2011 11

12. The Continuum (borked)
MaestroDev
Tuesday, February 22, 2011 12

13. Proof == Trust
MaestroDev
Tuesday, February 22, 2011 13

14. Qualty Assuance
• Product Control
– Requirements
– Change
– Release
• Catch “bugs” before presentation
– Functional AND Non-Functional
MaestroDev
Tuesday, February 22, 2011 14

15. Bi-Directional (required)
MaestroDev
Tuesday, February 22, 2011 15

16. QA Frontend
• Agile
– QA/Testers on
team
– Stories -> UAT
– Continuous
Integration
MaestroDev
Tuesday, February 22, 2011 16

17. QA Backend
• Test harness(es)
– Integration
– Functional
– Non-functional
MaestroDev
Tuesday, February 22, 2011 17

18. QA’s DevOp Priorities
• Testing must be a first-class citizen
MaestroDev
Tuesday, February 22, 2011 18

19. QA’s DevOp Priorities
• Testing must be a first-class citizen
“We’re not worried about
testing...our biggest
problem is automating
deployment”
MaestroDev
Tuesday, February 22, 2011 18

20. QA’s Priorities (cont.d)
• Release management
MaestroDev
Tuesday, February 22, 2011 19

21. Driving Trust via Releases
Results (Deliverables)
• Proof (Testing) Functional
Validated &
Versioned
(released)
Functional &
Non-Functional
Unit and Integration Integrated &
Libraries/Apps Deployed
(promoted)
MaestroDev
Tuesday, February 22, 2011 20

22. Dev-QA-Ops Choreography
MaestroDev
Tuesday, February 22, 2011 21

23. Push QA into Ops and Dev
• *DD
– Behavior DD (Agile QA?)
– Pulls non-functional from Ops
– Pushes non-functional to Devs
• Less Unit, Mo Integration, Less Mock
• Automate functional, but to a point
• Ops and QA enable Dev testing
MaestroDev
Tuesday, February 22, 2011 22

24. Push QA into Ops and Dev
• *DD
– Behavior DD (Agile QA?)
“BDD has reduced our
– Pulls non-functional from Ops
prerelease bugs by 80%,
– Pushes non-functional to Devs
in a side-by-side
• Less Unit, Mo Integration, Less Mock
• Automate functional, but to a point
comparo”
• Ops and QA enable Dev testing
MaestroDev
Tuesday, February 22, 2011 22

25. Alternative Perspective
• Building missiles
– Platform shall have a maximum range of
100km
MaestroDev
Tuesday, February 22, 2011 23

26. Validation and Verification
Functional
Libraries
Apps/Services
Functional &
Non-Functional
Integrated &
Modeling & Sims Deployed
Algorithms/Designs
MaestroDev
Tuesday, February 22, 2011 24

27. Quality Quality
• Tests must be good
– More to it than just assertions
• True negatives? False Positives?
– Testing for Failure
• Brittle Tests? Avoid testing content
• Changes should come with tests
(open sourcey way)
MaestroDev
Tuesday, February 22, 2011 25

28. Truth isn’t Good Enough
• Regressions (read blame)
– A single version change affects truth
• Measuring/Monitoring
– Earlier the better
• Trending (continuous improvement)
– Continuous improvement
MaestroDev
Tuesday, February 22, 2011 26

29. Dev/QA Reporting
MaestroDev
Tuesday, February 22, 2011 27

30. MaestroDev
Tuesday, February 22, 2011 28

31. DevOps
MaestroDev
Tuesday, February 22, 2011 29

32. Questions
http://devopsweekly.com/
MaestroDev
Tuesday, February 22, 2011 30