To assist in risk analysis, prioritization of testing, and test reporting (telling your testing story), you need a thorough Test Coverage Outline (TCO)—a road map of your proposed testing activities. By creating a TCO, you can prepare for testing without having to create a giant pile of detailed test cases. Paul Holland says that a comprehensive TCO helps the test team to get buy-in for the overall test strategy very early in the project and is valuable for identifying risk areas, testability issues, and resource constraints. Paul describes how to create a TCO including the use of heuristic-based checklists to help ensure you don’t overlook important elements in your testing. Learn multiple approaches for critical information gathering, the artifacts used as input for creating a TCO, and how you can use a TCO to maintain testing focus. Take back a new, lightweight tool to help you tell the testing story throughout your project.

1. W2
Test Techniques
5/1/2013 11:30:00 AM
The Test Coverage Outline: Your
Testing Road Map
Presented by:
Paul Holland
Testing Thoughts
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Paul Holland
An independent software test consultant and teacher, Paul Holland has more than sixteen years of handson testing and test management experience, primarily at Alcatel-Lucent where he led a transformation of
the testing approach for two product divisions, making them more efficient and effective. As a test
manager and tester, Paul focused on exploratory testing, test automation, and improving testing
techniques. For the past five years, he has been consulting and delivering training within Alcatel-Lucent
and externally to companies such as Intel, Intuit, Progressive Insurance, HP, RIM, and General Dynamics.
Paul teaches the Rapid Software Testing course for Satisfice. For more information visit
www.testingthoughts.com.

3. Product Coverage Outlines
and How to Use Them
Paul Holland
Test Consultant and Teacher
at Testing Thoughts
My Background
• Independent S/W Testing consultant since Apr 2012
• 16+ years testing telecommunications equipment and
reworking test methodologies at Alcatel-Lucent
• 10+ years as a test manager
• Presenter at, STAREast, STARWest, Let’s Test, CAST,
STARCanada, and EuroStar
• Keynote at KWSQA conference in 2012
• Facilitator at 25+ peer conferences and workshops
• Teacher of S/W testing for the past 5 years
• Teacher of Rapid Software Testing
– through Satisfice (James Bach): www.satisfice.com
• Military Helicopter pilot – Canadian Sea Kings
April, 2013
©2013 Testing Thoughts
2
1

4. Attributions
• Many of the concepts that I am presenting
come from the Rapid Software Testing
class developed by James Bach and
Michael Bolton
April, 2013
©2013 Testing Thoughts
3
Test Strategy Development
• A Test Strategy is often developed by:
– Following a template
– Envisioning the entire product as a whole
– Using elements from previous features/products
– Using estimates from the past
– Some use development estimates of complexity
and effort to determine test effort
April, 2013
©2013 Testing Thoughts
4
2

5. An Alternative
• Each feature/product is different from other
features/products
• Start with “raw” project specific information:
– Detailed description of the elements of the
feature/product
– Identify specific risks related to the project
– Develop a list of “test ideas”
April, 2013
©2013 Testing Thoughts
5
Product Coverage Outline
• A Product Coverage Outline can be
described as:
– A parts list
– An inventory of elements
– Components of a product
• Essentially a list of items of a project that can
be tested in some way
April, 2013
©2013 Testing Thoughts
6
3

6. Benefits of a PCO
• Helps eliminate tunnel vision
• Will help display your thoroughness to others
– Help you gain respect
• Will allow you to get “buy-in” at a high level of
what you are going to consider when testing
• Helps prioritize your testing
• Helps in reporting
April, 2013
©2013 Testing Thoughts
7
Product Coverage Outline
How to create a PCO
April, 2013
©2013 Testing Thoughts
8
4

7. Product Coverage Outline
• Take a tour of the feature/product
– Investigate every menu item
– List sub-menus
– Context specific items
– Text boxes
– Radio buttons / Check boxes
– Visual elements (pictures, background, logos)
– Pop-up boxes
April, 2013
©2013 Testing Thoughts
9
Product Coverage Outline
• Talk with designers, business people,
customers, customer service, etc.
• Look at previous versions
• Look at marketing information
• Read the functional specification document
(*although, I would do this later – seriously)
April, 2013
©2013 Testing Thoughts
10
5

8. Product Coverage Outline
• Consider the following (SFDIPOT):
– Structure of the program (smallest components)
– Functionality (individual features)
– Data (I/O, create, store, manipulate, backup,
– Interface (User interfaces, APIs,
)
)
– Platform (Computer, CPU, OS, browser,
)
– Operations (How is it used by customers)
– Timings (Race conds, time of day/wk/mth/yr,
)
(Taken from the Rapid Software Testing Course, Bach and Bolton)
April, 2013
©2013 Testing Thoughts
11
Product Coverage Outline
• As you are investigating the feature/product
– Create a list of ALL the elements that MAY need
to be tested in some way
• Make a mind map (I use XMind)
• Enter the elements into Excel (or a word processor)
• Create a list on paper, flip chart, or whiteboard
• Make a bunch of sticky notes
(I would encourage electronic versions for ease of
sharing and re-use – but it is not mandatory)
April, 2013
©2013 Testing Thoughts
12
6

9. Risk List
• Create a list of project specific risks
• These are in addition to “generic” risks
• Talk with designers, business managers,
customers, customer support, etc.
• Try to identify risks during the creation of the
Product Coverage Outline
April, 2013
©2013 Testing Thoughts
13
Test Ideas
• This is NOT a list of specific test charters
• List ways to test that you came across
during investigation. For example:
– Overflow input buffers
– Check interop between two components
– Verify information in text boxes/help screens
– Modify values, change all values from default
(These are only a very small set of examples)
April, 2013
©2013 Testing Thoughts
14
7

10. 3 Raw Lists: Now What?
• Use these three lists as the starting point
for your test strategy
• Get agreement from stakeholders of the
components and their importance
• They are important in “traditional” testing
environments as well as for agile teams
April, 2013
©2013 Testing Thoughts
15
3 Raw Lists: agile testing
Use these 3 lists to:
• Create your list of Charters
– Combine the test ideas, risks and components to
create specific charters
– Add “test ideas” under each Charter
• Help with the initial prioritization
• Assist in creation of new Charters during testing
• Help generate a better definition of done
April, 2013
©2013 Testing Thoughts
16
8

11. 3 Raw Lists: Traditional
• Create your list of Test Cases (shudder)
• If you must create scripted test cases,
using these three lists will help:
– Give you better coverage (not just using the
requirements)
– Reduce rework (because you toured SUT)
– Prioritize the test cases for execution and
reuse
April, 2013
©2013 Testing Thoughts
17
Hands on
Creation of a PCO for a simple program
• Lego MindstormsTM robot which changes
behavior depending on “sensed” color
• A Flash program that simulates the robot
• Generation of a PCO for the robot
• How would PCO differ for the Flash
simulation?
April, 2013
©2013 Testing Thoughts
18
9

12. April, 2013
©2013 Testing Thoughts
19
10