Key Takeaways - See what makes a good test strategy - Learn how to make a thorough test strategy - Identify what is the ‘Heuristic Test Strategy Model’ is - Develop a solid test strategy that fits fast - Discover how diversification can help you to create a test strategy

1. Practical Test Strategy
Using Heuristics
EuroStar Webinar, September 2016
Huib Schoots

2. Thanks to:
• Ruud Cox for the many discussions on this topic
• Jean-Paul Varwijk for helping me making it awesome
• Fiona Charles and Rikard Edgren for inspiration
• Obviously James Bach and Michael Bolton for sharing
their knowledge about Rapid Software Testing and the
Heuristic Test Strategy Model
Many slides are taken from Rapid Software Testing and are used with
permission. Rapid Software Testing is developed by James Bach and
Michael Bolton. Also see: http://www.satisfice.com/info_rst.shtml
Acknowledgements
2Practical Test Strategy Using Heuristics

3. What is test strategy?
Think about these questions:
1. What is test strategy to you?
2. Why do you make your test strategy?
3. What does your test strategy look like?
Test Strategy (according to ISTQB Glossary definition)
A high-level description of the test levels to be performed
and the testing within those levels for an organization or
programme (one or more projects).
3Practical Test Strategy Using Heuristics

4. Test strategy to me
4Practical Test Strategy Using Heuristics
Source: EuroStar Webinar Thinking Strategically About Testing by Fiona Charles

5. Creating a test strategy: problem solving!
1. Define the testing problems (or test missions)
2. Define solutions to these problems
3. Communicate / capture / execute
A test strategy is a solution to a complex problem:
How do we meet the information needs of the
stakeholders in the most efficient way possible?
5Practical Test Strategy Using Heuristics

6. Test strategy
Strategy: The set of ideas that guide your test design
Logistics: The set of ideas that guide your application of
resources to fulfilling the test strategy
Plan: The set of ideas that guide your test project
plan = strategy + logistics
6Practical Test Strategy Using Heuristics

7. Strategy – execution – reporting
Test Strategy Test ReportingTest Execution
Practical Test Strategy Using Heuristics 7

8. Testing
Strategy – execution – reporting
Practical Test Strategy Using Heuristics 8

9. A growing test strategy
Over time we learn & discover more
Practical Test Strategy Using Heuristics 9

10. Things to consider…
10Practical Test Strategy Using Heuristics
Source: Nordic Testing Days Workshop Test Strategy the next level by Rikard Edgren

11. Test strategy in agile…
US
Iteration 0 Hardening
Sprint
ReleaseIteration Test Strategy
Story Test Strategy Release Test Strategy
US
US
US
US
US
US
US
US
US
US
US
US
US
US
US
US
US
US
US
US
US
US
US
Sprint 1 Sprint 2 Sprint 3 Hardening
Sprint
Sprint 4 Sprint 5 Sprint 6
Overall Test Strategy
Release
Practical Test Strategy Using Heuristics 11

12. Tests
Project
Environment
Product
Elements
Quality
Criteria
Perceived
Quality
Heuristic Test Strategy Model
Practical Test Strategy Using Heuristics 12

13. Tests
Project
Environment
Product
Elements
Quality
Criteria
Perceived
Quality
Heuristic Test Strategy Model
Practical Test Strategy Using Heuristics 13

14. Heuristic Test Strategy Model
14Practical Test Strategy Using Heuristics

15. Project Environment
Ways to understand our context
15Practical Test Strategy Using Heuristics
MIDTESTD
§ Mission § Equipment & tools
§ Information § Schedule
§ Developer relations § Test items
§ Test team § Deliverables

16. Quality Criteria
Identifying value and threats to it…
16Practical Test Strategy Using Heuristics
§ Capabililty § Scalability
§ Reliability § Compatibility
§ Usability § Performance
§ Charisma § Installability
§ Security § Development
CRUCSS CPID

17. Coverage
Interesting coverage:
• Product coverage
• Risk coverage
• Requirements coverage
17
____ coverage is how thoroughly you have examined
the product with respect to some model of ____
Practical Test Strategy Using Heuristics

18. Product Elements
Want to cover the product?
18Practical Test Strategy Using Heuristics
SFDIPOT
§ Structure § Platform
§ Function § Operations
§ Data § Time
§ Interfaces

19. The General Test Techniques
“Ways to test…”?
19Practical Test Strategy Using Heuristics
FDSFSCURA
§ Function testing § Claims testing
§ Domain testing § User testing
§ Stress testing § Risk testing
§ Flow testing § Automatic checking
§ Scenario testing

20. Diversify!
20Practical Test Strategy Using Heuristics
http://www.satisfice.com/presentations/etta.pdf

21. Thirty-Four Test Strategy Heuristics
21Practical Test Strategy Using Heuristics
Mission
Information
Developer relations
Team
Equipment & tools
Schedule
Test Items
Deliverables
Structures
Functions
Data
Interfaces
Platforms
Operations
Time
Capability
Reliability
Usability
Charisma
Security
Scalability
Compatibility
Performance
Installability
Development
Function testing
Domain testing
Stress testing
Flow testing
Scenario testing
Claims testing
User testing
Risk testing
Automatic testing
Project
Environment
Product
Elements
Quality
Criteria
Test
Techniques

22. Balanced test strategy
Project
Environment
Practical Test Strategy Using Heuristics 22

23. Ask for testability!
23Practical Test Strategy Using Heuristics
• Controllability
• Observability
• Configurability
• Availability
• Simplicity
• Stability
• Information
Testing is far more rapid
when the product is more testable
Log files!
Scriptable
Interface!

24. Test strategy is …
24Practical Test Strategy Using Heuristics
Source: Nordic Testing Days Workshop Test Strategy the next level by Rikard Edgren

25. Make your test strategy visual!
25Practical Test Strategy Using Heuristics

26. Make your test strategy visual!
Practical Test Strategy Using Heuristics 26

27. Make your test strategy visual!
27Practical Test Strategy Using Heuristics

28. Questions?
!!!
???
Practical Test Strategy Using Heuristics 28

29. huib.schoots@improveqs.nl
@huibschoots
www.huibschoots.nl/blog
Practical Test Strategy Using Heuristics 29

30. References
30Practical Test Strategy Using Heuristics
• Rapid Software Testing – James Bach & Michael Bolton
http://www.satisfice.com/info_rst.shtml
• Heuristic Test Strategy Model – Designed by James Bach
http://www.satisfice.com/tools/htsm.pdf
• Heuristic Test Planning – James Bach
http://www.satisfice.com/tools/satisfice-cm.pdf
• Heuristic Risk-Based Testing – James Bach
http://www.satisfice.com/articles/hrbt.pdf
• Basics Revisited: Test Strategy - Fiona Charles
http://www.quality-intelligence.com/articles/BasicsRevisited-TestStrategy.pdf
• Webinar: Thinking Strategically About Testing - Fiona Charles
http://testhuddle.com/resource/thinking-strategically-about-testing-with-fiona-charles/
• What is a good test strategy – Rikard Edgren
http://thetesteye.com/blog/2013/09/what-is-a-good-test-strategy
• Software Quality Characteristics – Thetesteye.com
http://thetesteye.com/blog/2011/11/software-quality-characteristics-1-1/
• Workshop Test Strategy the next level – Rikard Edgren
http://nordictestingdays.eu/sites/default/files/NTD2014%20Presentations/TestStrategyNextLevel_FullDayTutorial.pdf
• Testability heuristics - James Bach
http://www.satisfice.com/tools/testable.pdf