I looked at my cards. 2 Aces. The best hand possible to have in poker on an empty board. At this point there is no risk that I can be beaten. I decide to exploit the situation. Get as much value as possible, but not letting my opponents know I have such a good hand. I don’t raise. 3 cards come on the board. I wait. The 4th card comes. I still wait. The fifth and last card comes and I make my move. I put all my money on the line and as it turns out, I got beaten by someone who has made a straight. How is this possible? I had the best hand, I evaluated the risk and still lost. The reason is obvious, the board changed 3 times, and with each extra card, my risk of losing also changed. And I did not adapt. I didn’t re-evaluate my risks and acted accordingly. There are quite a few games that deal with risks and risk responses. Poker and Monopoly are a few examples. There are world championships held in these games and there is general consensus who are the best players in the world. Those players have game tactics. What if we can map those tactics to Risk Based Testing? Can we improve our process based on those successful game tactics? In this presentation, I will elaborate on a few game tactics and map them on the Risk Based Testing process. I will give concrete examples of similarities between them and demonstrate that they can be adapted to improve our test process.

1. © 2011 CTG, Inc.
Playing around with
Risks
Jurgen Cleuren
Nov. 24th 2011

2. Introduction
• Projects are done in a probabilistic environment
– Incomplete information
– Parameters change over time
– What is true in the beginning of the project, can be false some time
later
• Games in a probabilistic environment
– Incomplete information (cardgames,…)
– Random element (dice, cards,…)

3. Which games ?
• Poker
• Monopoly
• Backgammon

4. Succes:
First learn the rules, then play better than everyone else
- Albert Einstein -
• Rules -> requirements
• Every game starts with learning and agreeing on the rules -> every
project starts with defining and agreeing on the requirements

5. TEXAS HOLD‟EM POKER

6. Rules of Texas Hold‟em
• 2 Blind cards
• Betting round
• 3 Open Community cards (flop)
• Betting round
• 1 Open Community card (Turn)
• Betting Round
• 1 Open Community Card (River)
• Betting Round
• Best hand of 5 cards wins

7. Hand Ranking
• Highest Card
• Pair
• 2 Pair
• 3 of a kind
• Straight (5 consecutive cards)
• Flush (5 cards of the same suit)
• Full House (3 and 2 )
• 4 of kind
• Straight Flush (5 consecutive cards of the same suit)

8. General test flow vs Poker round
• Create FTT/Risk Matrix
• Software Delivery
• Full functional test
• Bugfixing
• New Software Delivery
• Retesting and regression
• Bugfixing
• Final Software Version
• Regression Test
• Get Blind cards – evaluate risk - Bet
• Flop (3 cards)
• Re-evalute hand and Risk
• Betting round
• Turn (1 card)
• Re-evaluate hand and Risk
• Betting round
• River (1 card)
• Final bets and showdown

9. Similarities / Differences
3 Recurring phases
1 big phase (Complete Functional testing vs Flop)
2 small phases (retest + regression vs Turn + River)
Determine Risk before 1st test run (Risk Matrix or FTT vs 1st bet)
Adapt Risk Matrix or FTT after each test run

10. Should the FTT or Risk Matrix be adapted ?
• Every Test run gives new information
• Likelihood of risks change
– Failed test cases get a higher likelihood
– Passed test cases in unchanged code have a lower likelihood
• Closer to deadline => Risks can change
• Reporting is more clear
– Each Risk Matrix/FTT tree is a snapshot of the project at a certain time

15. Who ?
• Test manager should take the lead
• Preferably Project Board
• Test manager can do it by himself, but the board should at least be aware
that this activity is done

16. Justification
• A Poker hand needs justification at any point in the game
– Having the best hand in the beginning doesn‟t imply that you are going
to win
– You must be prepared to fold when you are not winning anymore
– The money you‟ve already bet doesn‟t count
• It is in the pot so it is not your money anymore
• Don‟t put more money in a losing hand
– No justification anymore: get out of the hand
– Cut your losses

17. Project justification
• A project needs justification at any point in time
• During testing: justification after each test run
– New information is given
– Risks are updated
• No Justification means project should be closed
• Previous investments do not count
– Don‟t put more money in a failing project
• Cut your losses

18. Test process justification
• Get Blind cards – evaluate risk - Bet
• Flop (3 cards)
• Re-evalute hand and Risk
• Betting round
• Turn (1 card)
• Re-evaluate hand and Risk
• Betting round
• River (1 card)
• Final bets and showdown
• Create FTT/Risk Matrix
• Software Delivery
• Full functional test
• Justification
• Bugfixing
• New Software Delivery
• Retesting and regression
• Justification
• Bugfixing
• Final Software Version
• Regression Test
• Justification

19. Result Oriented Thinking
• A decision can be right even if the outcome is not favourable
• Focus more on the decision and the „why‟ instead of the result
– Tester A completes a test set in 4 days by skipping tests
– Tester B completes the same test set in 6 days through thoroughly
testing
– No defects were discovered in production
– Who would you reward the most ?

20. Thought process
• In poker, to anticipate and understand your opponents actions, you need
to think as your opponent and not as you in his situation
• What motivates you, does not necessarily motivate another person
• Successful people ask better questions
– WHY? is more important than HOW? or WHO?
“Someone who knows HOW will always have a job
Someone who knows WHY will always be his boss”

21. Luck
“ Luck is when preparation meets opportunity”
- Seneca –
• Be prepared to get lucky
– In poker, sometimes you need to get lucky. When you get lucky, be sure
to take a big pot out of it.
– Sometimes a best case scenario happens, but we need to take
advantage of it.
– Being ahead of planning is more valuable if you can actually do
something with this situation

22. MONOPOLY

24. Aspects of the game
• Investing in houses
• The higher the investment, the bigger the payoff
• Some spaces have a higher probability
• Random element: dice

25. Analogy to Risk Based Testing
• Different probability of landing on spaces = Likelihood
• Higher Payoff = Impact
• What can monopoly teach us ?
– What is more important: Impact or Likelihood ?

26. Impact and likelihood on the board

27. Winning Monopoly strategies
• Complete Orange Colour group and invest as soon as possible
– Why Orange ? It has the biggest probability of other players landing on
it
– Be prepared to even trade down to get this colour group
• Complete Red Colour group as a second priority
– Why Red ? It has the second biggest probability of other players
landing on it

28. What lessons are to be learned
• Likelihood >>>>>> impact
– The weight of Likelihood should be > 50%
– The weight of Impact should be < 50%

29. BACKGAMMON

30. Backgammon

31. Important rules of Backgammon
• Goal: get all your tiles from one end to the other
• Only tiles that are standing alone on a pillar can be captured
• A captured tile has to be brought back in play at the beginning of the
board
• Random element: Dice

32. Translation to risks
• Your own checkers: Risks
• Opponents checkers: Causes
• Whenever one of your checkers is captured it‟s in fact a cause hitting a
risk
• A risk is mitigated when the checker cannot be captured (2 or more on
one pillar)

33. Translation of risk priority
• Low risk: checker in the first quadrant
• Medium risk: checker in the second or third quadrant
• High risk: checker in the fourth quadrant

34. What can Backgammon teach us?
• Which risks should be mitigated and which are low priority ?
• There might be a possibility to remove a cause, but in the same way
creating a new risk. Should we do it ?
– Eg: Back-up testmanager vs full time testing

35. Backgammon Strategies
• Checkers in the 1st quadrant don‟t have to be protected. Moving forward
is the better play.
Risks with low priority don‟t have to be mitigated. The correcting cost is
usually way less than the mitigation costs
• Checkers in the 4th quadrant need to be protected at all costs.
Risk with high priority need to be mitigated at all costs
• For checkers in the 2nd and 3rd following rule applies: Always take a
chance to capture
If you can remove a cause and therefore create a medium of low risk,
do so

36. CONCLUSIONS

37. Conclusions
• What Poker taught us:
– Adapt FTT/Risk tree after each test run
– Priorities of test items can change
– Justification has to be done after each test run
– Don‟t focus on results, focus on decisions
– Be prepared to get lucky!

38. Conclusions
• Create FTT/Risk Matrix
• Software Delivery
• Full functional test
• Bugfixing
• New Software Delivery
• Retesting and regression
• Bugfixing
• Final Software Version
• Regression Test
• Create FTT/Risk Matrix
• Software Delivery
• Full functional test
• Adapt FTT/Risk Matrix
• Justification
• Bugfixing
• New Software Delivery
• Retesting and regression
• Adapt FTT/Risk Matrix
• Justification
• Bugfixing
• Final Software Version
• Regression Test
• Adapt FTT/Risk Matrix
• Justification

39. Conclusions
• Monopoly
– Likelihood >>>>> Impact
• Backgammon
– Don‟t mitigate low priority risk
– Always mitigate high priority risks
– Removing a cause and creating a medium or low risk is the way to play

40. QUESTIONS AND ANSWERS
Jurgen Cleuren (jurgen.cleuren@ctg.com)