Visit SoftTest Ireland www.softtest.ie and sign up for access to free Irish Software Testing events.

1. Presented by Stephen Allott CaseMaker International A Diaz & Hilterscheid group company Effective Testing Using industry standard techniques SoftTest Ireland 16 th February 2006

3. Why test software?

4. Read all about it! US site – down for 3 days, forced to return $30 fee to nearly 260,000 online “fantasy baseball” players. US site down for 2 days, $280,000 in lost sales. Shopping Site: 6-hour downtime: $400,000 in direct costs plus additional in discounts to inconvenienced customers. UK - Concerns over bugs and delays in new air traffic control software UK – Payment clearing system failure at month end delayed wages to many customers UK - Televisions, Cameras sold at very low prices

12. White Box Techniques

22. Example – Train Tickets Pay £30 Pay £20 Pay £15 Pay £10 Actions N Y N Y Travels after 9 am N N Y Y Holds a railcard Rule 4 Rule 3 Rule 2 Rule 1 Conditions

23. Example – Train Tickets Test Cases Pay £30 Pay £20 Pay £15 Pay £10 Expected results N Y N Y Travels after 9 am N N Y Y Holds a railcard Test 4 Test 3 Test 2 Test 1 Inputs

24. Example – ATM Reject Reject Reject Auth Actions N Y N Y Enter valid PIN N N Y Y Enter valid card Rule 4 Rule 3 Rule 2 Rule 1 Conditions

25. Example – ATM “don’t care” Reject Reject Auth Actions --- N Y Enter valid PIN N Y Y Enter valid card Rule 3 Rule 2 Rule 1 Conditions

26. Example – File Backup Y Y Y N Stick inserted Works Fails Fails Fails Actions Y N --- --- Sufficient space Y Y N --- Stick formatted Rule 4 Rule 3 Rule 2 Rule 1 Conditions

27. State Transition Testing Source: Standard for software component testing, working draft 3.3 http://www.rmcs.cranfield.ac.uk/~cised/sreid/BCS_SIG/index.htm.

28. State Transition Testing Source: Standard for software component testing, working draft 3.3 http://www.rmcs.cranfield.ac.uk/~cised/sreid/BCS_SIG/index.htm.

29. 0-Switch coverage Source: Standard for software component testing, working draft 3.3 http://www.rmcs.cranfield.ac.uk/~cised/sreid/BCS_SIG/index.htm. S2 S4 S1 S1 S3 S2 Finish State D AD T T AT D Expected Output DS R CM TS R CM Input S4 S2 S2 S3 S1 S1 Start State 6 5 4 3 2 1 Test Case

30. 1-Switch coverage Source: Standard for software component testing, working draft 3.3 http://www.rmcs.cranfield.ac.uk/~cised/sreid/BCS_SIG/index.htm. S4 S1 S2 S3 S2 S3 S2 S1 S4 S1 Finish State AD T D AT D AT D T AD T Exp. Output R CM DS R CM R CM TS R CM Input S2 S2 S4 S1 S1 S1 S1 S3 S2 S2 Next State D D AD T T T T AT D D Exp. Output DS DS R CM CM TS TS R CM CM Input S4 S4 S2 S2 S2 S3 S3 S1 S1 S1 Start State 10 9 8 7 6 5 4 3 2 1 Test Case

31. Pairwise Testing Acknowledgement This session on pairwise testing techniques is based on the book by Lee Copeland: A practitioner’s guide to software test design, published by Artech House, 2004, ISBN 1-58053-791-X

35. Orthogonal Array L 9 (3 4 ) This orthogonal array has 9 rows, 4 columns and three ‘levels’ per column 1 2 3 3 Row 9 3 1 2 3 Row 8 2 3 1 3 Row 7 2 1 3 2 Row 6 1 3 2 2 Row 5 3 2 1 2 Row 4 3 3 3 1 Row 3 2 2 2 1 Row 2 1 1 1 1 Row 1 D C B A

36. Orthogonal Array L 9 (3 4 ) Each variable becomes a column in the orthogonal array 1 2 3 3 Row 9 3 1 2 3 Row 8 2 3 1 3 Row 7 2 1 3 2 Row 6 1 3 2 2 Row 5 3 2 1 2 Row 4 3 3 3 1 Row 3 2 2 2 1 Row 2 1 1 1 1 Row 1 When Hotel Price Where

37. Orthogonal Array L 9 (3 4 ) For each column we fill out the possible choices 1 2 3 Europe Row 9 3 1 2 Europe Row 8 2 3 1 Europe Row 7 2 1 3 US Row 6 1 3 2 US Row 5 3 2 1 US Row 4 3 3 3 UK Row 3 2 2 2 UK Row 2 1 1 1 UK Row 1 When Hotel Price Where

38. Orthogonal Array L 9 (3 4 ) Now the array is complete Now 4 star High Europe Row 9 Later Basic Medium Europe Row 8 Soon Luxury Budget Europe Row 7 Soon Basic High US Row 6 Now Luxury Medium US Row 5 Later 4 star Budget US Row 4 Later Luxury High UK Row 3 Soon 4 star Medium UK Row 2 Now Basic Budget UK Row 1 When Hotel Price Where

39. Orthogonal Array L 9 (3 4 ) Each row becomes a test case – you can add more if you wish Now 4 star High Europe Test 9 Later Basic Medium Europe Test 8 Soon Luxury Budget Europe Test 7 Soon Basic High US Test 6 Now Luxury Medium US Test 5 Later 4 star Budget US Test 4 Later Luxury High UK Test 3 Soon 4 star Medium UK Test 2 Now Basic Budget UK Test 1 When Hotel Price Where

41. What size array do we need? How many possible combinations are there? Choices Variable NT, XP, Linux Server O/S IIS, Apache, WebLogic Server W95, W98, W2000, NT, XP, XP-PRO Operating systems None, Real player, Media Player Plug-ins IE5, IE6, …. 8 in total Browsers

46. After all, what is quality?

47. Presented by Stephen Allott CaseMaker International A Diaz & Hilterscheid group company Effective Testing Using industry standard techniques SoftTest Ireland 16 th February 2006