The document discusses the concept of patterns in testing and their usefulness. It outlines several types of patterns in testing, including data patterns, test patterns, defect patterns, and management patterns. It also suggests that identifying patterns can help predict future defects and failures. Overall, the document advocates that recognizing patterns in testing data can provide insights and help improve the testing process.
3. Patterns in real world
Design Patterns and Test Patterns
Usefulness of patterns in testing
Patterns in testing
Data Patterns
Test Patterns
Defect Patterns
Test management Patterns
People management Patterns
Control Patterns
What we do not know today
What we might need to create – pattern language
3
6. The sum of the
numbers in any row
is equal to 2n
Source - http://ptri1.tripod.com/
6
7. How do you learn your native language?
How do you learn your second language?
7
8. The brain is a pattern-mad supposing
machine. Given just a little stimuli, it divines
the probable. …it recognizes familiar patterns
and acts with conviction.
As much of a confection as our mental maps
are, they (Patterns) allow us to
speculate, rehearse and make plans.
*http://www.nytimes.com/2004/06/15/science/essay-i-sing-the-body-s-pattern-recognition-machine.html
8
9. *A form or model proposed for imitation
*Something designed or used as a model for
making things
Design pattern - general reusable solution to
a commonly occurring design problem
From MW Dictionary*
9
10. I predict that one of the future breakthrough
in testing will come through pattern
recognition
Example
Patterns of defects arising out of patterns of
errors in code/design/requirements
Patterns of project failures arising out of data
collected during project execution
10
11. Patterns exists everywhere - in
defects, code, design, processes
Pattern identification helps predict the next
set of events in the chain enabling you to do
risk mitigation
Patterns enable you to distinguish the woods
from the trees
11
12. Data Patterns
Test Patterns
Defect Patterns
Test management Patterns
People management Patterns
Control Patterns
12
13. Ross Collard listed a number of data patterns
for functional and performance testing -
Routine Live Data
Baseline
Batch Volume or Parallel
Benchmark
Pristine
Truncated
Minimally Redundant etc.
13
14. People often work on one side of boundary
while testing
Make them work on both sides/all sides of
boundary
Example –
Triangle Problem negative test cases - no value, 1
value, 2 values, all 3 values
what about the other side of the boundary - more
number of values?
14
15. Patterns can help uncover new test design
techniques OR new ways to look at existing
test design techniques
Apply test design techniques to create tests
Monitor the bugs arising OUT(SIDE) of tests
Establish the bugs that could have been found
using existing techniques
Defects not “findable” using techniques should be
analyzed for patterns and a technique using which
these could be found
15
16. Some patterns in type of bugs not found using
tests created using techniques
Interaction of application with the operating
system
Reason – Design specifications hardly go to that level
of detail and system specifications never
Defects related to interdependency of features
and code structures causing one portion of code
to be bound to another piece of code
Reason – Code dependencies are either not called out
explicitly, born in either assumptions or ignorance
(most of the time)
16
17. Associations
Contains
User Inputs
A model System Inputs Output Data
Entity under test
based Input Data
Output Actions
technique Input Actors
On other
Resembles elements/
entities
Environment (Ext. State)
state
State (Internal State)
diagrams On OS - Disk
- File
- Process
- Screen IO
- System Messages
Transient /Stable State
Parameters & their attributes
Parameters
State of Parameters
17
18. How many bugs do you find every month?
How many of them are type –
UI, Feature, cross-
feature, database, security…
Can you sub-classify them?
Do you know what are other testers finding?
Do you see any patterns in
What type of bugs you find
What types of bugs you don’t find?
18
19. If you find predictable bugs (based on your
past experience and pattern recognition) can
there be pattern recognition
for code?
Design?
Requirements?
19
20. Are there testers who do good bug isolation?
Are there those who do not?
How do they fail? What things they do not
do?
What patterns exist?
If I know those patterns can I improve their
skills?
20
22. Date Action Issue
24-Jan-01Clear work division in overlapping areas Performance test cases being written by Joe and Jane.
and ensuring communication between Test were duplicated. Some missing test cases
the overlapping/concerned people. because every body was not aware hence did not
review/contribute to the tests.
28-Jan-01Knowledge of the system (WES, WAS Some members of the team did not know what they
performance, tuning, deployment) had to do, others did not have required technical
should be available to the team. knowledge
14-Feb-01Communication of the documents, The release meeting with developers happened
Agenda, Minutes to the testing team without a copy of test plan (esp. dates) being
should be formal, via mail. available.
9-Mar-01User Guides esp. “how to get started” Swami sent a mail asking how to use the ABC app. We
should always accompany a build had to write some steps and send him the document.
What pattern do you see here?
This pattern emerges from long term view - The four
entries here are from a list of 133 entries between Jan and April 2001.
How is pattern recognition different from Root cause analysis?
22
23. Example – Patterns while Interviewing
assessing people, forming
impressions, validating, creating and refining
patterns
Remember - you have not discovered the last
pattern hence you may be wrong
Keep updating your impression of patterns and be
ready for the worst
23
24. Example – “Only a Guest” pattern
when people (are about to) leave the organization
without information to the management there
are some behavioral patterns that manifest
themselves.
Indifference, extra-hard work, sudden forgiveness
and not picking up of battles, unplanned
absenteeism, putting all paper-work in place etc.
24
26. Unified Modeling Language – Unified Testing
Language
No guarantee of success and no predictions
of failure
26
27. Applying patterns requires
Ignoring the small and looking at the large
Large amount of data (usually)
Reduction in the number of variables
Elimination of outliers
Pattern of outliers
Variations and taking cognizance of those
Variations and eliminating them
27
Root-cause analysis is one step in pattern recognition but not the only one RC – Communication/Training whereas pattern is things not getting done properly