When teams move towards continuous delivery and deployment, how do they manage the manual stages in their deployment pipeline? This talk gives some techniques to visualize pipelines, identify bottlenecks, find ways to remove them.

1. @lisacrispin
Lisa Crispin
With material from Abby Bangser, Ashley Hunsberger, Lisi Hocke, Janet Gregory,
& more
Copyright 2020 Lisa Crispin
Get Testing Bottlenecks Out of Your
Deployment Pipelines

2. @lisacrispin
A little about me
Quality Owner at
OutSystems, helping to
build observability
lisa@agiletester.ca
With Janet Gregory
New!

3. @lisacrispin
Talking about:
● Succeed with continuous delivery or deployment
● With confidence that nothing breaks
● No testing bottleneck
● It’s a whole-team effort

4. @lisacrispin
Continuous Delivery, 2010
Chapter 5: forms of testing common to all projects
include manual test stages which “… assert that the
system is usable and fulfills its requirements, detect any
defects not caught by automated tests, and verify that it
provides value to its users. These stages might typically
include exploratory testing environments, integration
environments, and UAT (user acceptance testing).”

5. @lisacrispin
Manual test stages, per Continuous Delivery:
● Common to all projects
● Assert system is usable and fulfils requirements
● Detects defects not found by automated tests
● Verify value provided to users
● Examples: Exploratory testing, integration
environments, and UAT

6. @lisacrispin
They’re part of the pipeline
But often forgotten…
Story time!

7. @lisacrispin
Let’s get visual

8. @lisacrispin

9. @lisacrispin

10. @lisacrispin
Visualize it – with your team
● Use virtual cards or stickies for pipeline stages
● Where are the bottlenecks?
● What can you try?
Photo by Llewellyn Falco from a team
pipeline visualization session

11. @lisacrispin
More testing bottlenecks
● Flaky tests
● Long-running test suites
● Unnecessary automated tests

12. https://github.com/ahunsberger/TestSuiteDesign - Ashley Hunsberger

13. @lisacrispin
Get the team together!
● Use the Test Suite Canvas to guide conversations
● Can you automate more of the now-manual tests?
● Can you get rid of any tests?

14. @lisacrispin
Conversations needed…
How do we fit testing activities
into continuous
delivery/deployment –
without slowing our pipeline

15. @lisacrispin
A few resources
● Agile Testing Condensed, Agile Testing and More Agile Testing,
Lisa Crispin and Janet Gregory, https://agiletester.ca
● Continuous Delivery by Jez Humble and David Farley,
https://continuousdelivery.com
● A Practical Guide to Testing in DevOps by Katrina Clokie
https://leanpub.com/testingindevops
● Accelerate by Dr. Nicole Forsgren, Jez Humble, Gene Kim
● Ashley Hunsberger’s Test Suite Canvas
https://github.com/ahunsberger/TestSuiteDesign