From your own experience it might not come as a surprise that most of today’s testing is unhelpful, filled with unnecessary paper work and folkloric activities. For some reason testing work often does not seem to be very helpful in projects. That is definitely a problem. If you are a tester, your manager might ask you for metrics that don’t make sense to you. And since you are a smart person, you have probably once in a while gamed the system. All that is certainly damaging to the industry. What can you do? This session brings you insight into Complexity Thinking with Dave Snowden’s Cynefin model and ties that to your job as a software tester. It offers you a way to look at software testing from a complexity thinking standpoint of view and gives you tools to argue your case if you are exposed to dysfunctional project settings. In addition to that, we will have some fun with idiotic metrics and to lighten up the serious topic we’ll engage in hilariously entertaining real life examples of bad metrics. To round it up, we’ll propose more meaningful alternatives.

1. Complexity Thinking,
Cynefin & Why Your
Testing Metrics All Suck
Big Time

2. Complexity Thinking, Cynefin &
Why All Your Testing
Metrics All
Sucks Big Time

3. Who Am I?
Who – Problem – The Model –Application – Metrics – Action – End

4. llari Henrik Aegerter
Managing Director – House of Test
VP of Marketing – Association for Software Testing
Beer Brewer
@ilarihenrik
www.houseoftest.ch
Who – Problem – The Model –Application – Metrics – Action – End

5. Disclaimer:
Keep in mind that I am a
*%§!$ Consultant
Question everything I say!
Who – Problem – The Model –Application – Metrics – Action – End

6. What is the problem?
Who – Problem – The Model –Application – Metrics – Action – End

7. We are applying methods in
software testing we wouldn’t in
comparable other situations
Who – Problem – The Model –Application – Metrics – Action – End

8. Who – Problem – The Model –Application – Metrics – Action – End

9. The Cynefin Model
Who – Problem – The Model –Application – Metrics – Action – End

10. Confusion between
Complicated vs. Complex
Complicated:
Degree to which something is difficult to
understand
Complex:
Degree to which Cause and Effect
are deterministic
Who – Problem – The Model –Application – Metrics – Action – End

11. Who – Problem – The Model –Application – Metrics – Action – End

12. Ordered
Linear
Complex
Non-Linear
Who – Problem – The Model –Application – Metrics – Action – End

13. How to apply Cynefin in
Testing
Who – Problem – The Model –Application – Metrics – Action – End

14. When confronted with a
testing task, ask yourself
how to act
Who – Problem – The Model –Application – Metrics – Action – End

15. Obvious
Sense - Categorize – Respond
World Expample:
Traffic Light
Testing Example:
Checking GUI Elements
Who – Problem – The Model –Application – Metrics – Action – End

16. Complicated
Sense - Analyze – Respond
World Expample:
Building a Space Ship
Testing Example:
Analysis of non-trivial
calculations
Who – Problem – The Model –Application – Metrics – Action – End

17. Complex
Probe - Sense – Respond
World Expample:
A roundabout
Testing Example:
Almost everything in testing,
Unknown unknowns, Exploratory
Testing
Who – Problem – The Model –Application – Metrics – Action – End

18. Safe to Fail Experiments
Who – Problem – The Model –Application – Metrics – Action – End

19. Chaos
Act - Sense – Respond
World Expample:
Kids
Testing Example:
A project in disarray
Who – Problem – The Model –Application – Metrics – Action – End

20. Some words about metrics
Who – Problem – The Model –Application – Metrics – Action – End

21. Cem Kaner on Metrics and
Measurements
http://kaner.com/pdfs/PracticalApproachToSoftwareMetrics.pdf
Who – Problem – The Model –Application – Metrics – Action – End

22. A General Problem with Metrics
Question is about Quality -
Measurements are Quantitative
Hence your measurement is a
surrogate one
Who – Problem – The Model –Application – Metrics – Action – End

23. Measurement Distortions
and
Goal Replacements
Who – Problem – The Model –Application – Metrics – Action – End

24. Who – Problem – The Model –Application – Metrics – Action – End

25. Examples of useless Metrics
- Number of Test Cases
- Number of Bugs found
- Degree (in %) of Automation
Example: Percentage of manual TC
”transformed” into Automation
→ coverage is always according to a model,
not the software itself
- ““““Test Progress” i.e. “number of “test
cases” “executed””””
Who – Problem – The Model –Application – Metrics – Action – End

26. Who – Problem – The Model –Application – Metrics – Action – End

27. Who – Problem – The Model –Application – Metrics – Action – End

28. Who – Problem – The Model –Application – Metrics – Action – End

29. Who – Problem – The Model –Application – Metrics – Action – End

30. Here’s a useful metric:
Number of managers fired for
asking their employees to
follow idiotic metrics
Who – Problem – The Model –Application – Metrics – Action – End

31. Here’s a useful metric:
Number of managers fired for
asking their employees to
follow idiotic metrics, per
month
Who – Problem – The Model –Application – Metrics – Action – End

32. Here’s a useful metric:
Number of managers fired for
asking their employees to
follow idiotic metrics, per
month, per tester
Who – Problem – The Model –Application – Metrics – Action – End

33. Call to Action
Who – Problem – The Model –Application – Metrics – Action – End

34. Study Cynefin to get a
better understanding of
complexity
Who – Problem – The Model –Application – Metrics – Action – End

35. Keep in mind that most of
testing belongs in the
complex area
Who – Problem – The Model –Application – Metrics – Action – End

36. Push back on invalid
demands
Who – Problem – The Model –Application – Metrics – Action – End

37. Good Luck on Your
Journey!
Who – Problem – The Model –Application – Metrics – Action – End

38. And Now Your Questions
Thanks!
@ilarihenrik
www.houseoftest.ch
ilari.aegerter@houseoftest.ch
Who – Problem – The Model –Application – Metrics – Action – End