1. Enabling Digital Transformation
Through DevOps
Gerie Owen & Peter Varhol
DevOps Days Indianapolis
7/25/2019

2. Gerie Owen
2
• Testing Strategist & Evangelist
• Sr. Test Manager at Cubic
• Subject expert on testing for
TechTarget’s
SearchSoftwareQuality.com
• International and Domestic
Conference Presenter
Gerie.owen@gerieowen.com

3. Peter Varhol
• International speaker and writer
• Degrees in Math, CS, Psychology
• Technology communicator
• Former university professor, tech journalist
• Cat owner and distance runner
• peter@petervarhol.com

4. Agenda
• What is Digital Transformation
• Synergies between Digital Transformation and DevOps
• Enabling Digital Transformation through Continuous Delivery
• Developing a Test Strategy for Continuous Testing

5. What is Digital Transformation?
• A fundamental shift in the way an organization approaches its
business and operational processes by using technology to increase
focus on and improve customer experience to increase business
value.

6. The Focus of Digital Transformation vs. DevOps
Digital transformation:
• Using technological innovation to:
• Increase business value through enhanced customer experience
• Streamline operational processes and business models and methodologies
DevOps:
• Enables innovation in technology and operations
• Increasing business value through enhanced customer experience
• Streamlining processes and eliminating bottlenecks

7. The Synergies
DevOps principles described in the Three Ways:
• Systems thinking,
• Amplifying feedback loops and
• Creating a cultural of experimentation and continual learning
These principles can be the basis for creating the major changes that
are required for digital transformation.

8. The Synergies
• Systems Thinking
• Focuses on the performance of an entire process is the most effective
approach to transforming business models and operational processes.
• Eliminates silos in both IT and business operations
• When changes are made to fundamental operations in siloed initiatives, they will likely
fail.
• Enables innovative, fundamental shifts in business workstreams through
technology.

9. The Synergies
• Amplifying feedback loops
• Is critical to digital transformation so that corrections can be made early and
often.
• Focuses on responsive, efficient and effective customer service to all internal
and external customers

10. The Synergies
• Creating a culture of continuous experimentation and learning
• Encourages innovation without fear of consequences if a new business,
process or technological approach fails.
• Taking risks and pushing the limits leads to true, ground-breaking innovation.

11. Enabling Digital Transformation throughout Continuous Delivery
Amplify
Feedback
Loops
Systems
Thinking
Culture of Continuous
Learning
Customer-
Centric Focus
Business Value
Focus

12. Enabling Digital Transformation Through Continuous Testing
• Enables digital transformation throughout the continuous delivery
pipeline by ensuring the customer-centric focus
• A focus on Quality at every stage of the Continuous Delivery process.
• Manages risk by increasing the effectiveness of our test processes.
• Assessing and mitigating business risk is the primary goal.

13. • Establishes a safety net that helps the DevOps team protect the user
experience.
• Requires a stable test environment to be available on demand.
• Integrates seamlessly into the software delivery pipeline and DevOps
toolchain.
• Delivers actionable feedback appropriate for each stage of the
delivery pipeline.
Enabling Digital Transformation Through Continuous
Testing

14. Components of a Continuous Testing Strategy
1. Streamline the Test Process
2. Champion Team Responsibility for Quality
3. Increase Velocity AND Improve Quality
4. Testing throughout the Build and Delivery Pipeline
5. Implement Automated Test Checkpoints
6. “Shift-Left” both Functional and Non-Functional Testing
7. Implement Continuous Monitoring in Test and Production

15. Optimize Test Cases
• Tests should be focused on the functionality they are validating.
• Tests should be as simple as possible; the more complex the test, the
more validation is required for the test script itself.
• Even though service virtualization can be used to fill in for missing
components in integration testing, tests should as independent as
possible.

16. Run Automated Tests In Parallel
• Speed up Automated Test Suites by running them in parallel on
multiple virtual machines; best practices include:
• Atomic Tests
• 1 test per test script
• Autonomous Tests
• Test should not rely on the results of previous test to run
• Quick-running Tests
• Keep execution time to one minute on the local environment
• Data Management
• Each Test should create and delete its own data

17. Identify and Mitigate Bottlenecks
• They Impede Velocity!
• An on-going process

18. Increase Velocity
• “Shift Left”; Push automation to the base of the testing pyramid.
• TDD, BDD
• Pair testers and developers
• Include risked-based manual and exploratory testing.
• Implement continuous functional & non-functional testing including:
• Performance
• Security
• Availability
• Accessibility
• Amplify automated feedback loops to enable risk-based decision-
making.

19. Improve Quality
• Improve quality
• Developers need to embrace testing.
• Show them what’s in it for them.
• Entire team is responsibility for quality.
• Implement post-production monitoring.
• Implement testing in production, based on application risk.

20. Integrate Testing Throughout the Delivery Pipeline
• Integrate automated tests in each stage the Continuous Integration
Pipeline.

21. Implement Automated Test Checkpoints
• Unit tests verify code.
• Component tests verify key features.
• Integration test verify workflows involving multiple applications.
• Risk-based, Optimized Regression tests run prior to each deployment.
• Post production health checks verify applications are up and running after
deployment.
• Production monitoring finds defects and performance issues before they
impact the customer.

22. “Shift Left” Functional and Non-Functional Testing
• Functional:
• Testers are involved with backlog grooming; ATDD
• TDD, BDD
• Automate early and often; embrace the Test Automation Pyramid
• Non-Functional
• Begin Performance testing as early as possible in order to correct design flaws
more easily.
• Security Testing
• Automated tests to ensure that code written in a secure manner
• Test execution security as early as possible

23. Continuous Monitoring in Test and Production
• Continuous Monitoring
• Comes from the DevOps principle of amplifying feedback loops in both test and
production.
• Enables collection and analysis of data and metrics throughout the lifecycle that are
used to optimized user experience.
• The most important data is about how users are interacting with the application and
feedback from customers. This data allows stakeholders and DevOps teams to make
targeted adjustments such as:
• Functional enhancements to remove customer pain points
• Upgrades to environments to improve application performance
• Testing in production includes:
• Monitoring data and transactions
• Simulating transactions
• Experimentation with real users
• A/B Testing.

24. Summary
• DevOps principles can be the basis for creating the major changes that
are required for digital transformation.
• The synergies between Digital Transformation are found in the Three
Ways
• Systems Thinking
• Amplifying Feedback Loops
• Creating a culture of continuous experimentation and learning
• Enabling Digital Transformation through Continuous Delivery involves
applying the synergistic focuses of Digital Transformation and
Continuous Delivery across the CD pipeline with Continuous testing.
• Continuous Testing enables digital transformation throughout the CD
pipeline by ensuring the customer-centric focus.

25. References
• Continuous Testing for IT Leaders, Wayne Ariola and Cynthia Dunlop
• A Practical Guide to Testing in DevOps, Katrina Clokie
• What Testers Need To Know About Continuous Testing, Wayne Ariola
• The Forrester Wave™: Modern Application Functional Test
Automation Tools, Q4 2016, Diego Lo Guidice
• http://itrevolution.com/the-three-ways-principles-underpinning-
devops/, 2012, Gene Kim

26. Enabling Digital Transformation
Through DevOps
Questions?