Test Automation Patterns

nt Session

Presented by:
Dorothy
Indepen ultant

Brought to you by:

340 Corporate Way, Suite Orange Park, FL 32073
888‐2
W3
Concurre
4/9/2014
10:30 AM

“Test Automation Patterns”

Graham
dent Testing Cons

300,
68‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

Dorothy Graham
Independent Test Consultant

In software testing for forty years, Dorothy Graham is coauthor of four books—
Software Inspection, Software Test Automation, Foundations of Software Testing and
Experiences of Test Automation—and is currently working with Seretta Gamba on a
new book on test automation patterns. A popular and entertaining speaker at
conferences and seminars worldwide, Dot has attended STAR conferences since the
first one in 1992. She was a founding member of the ISEB Software Testing Board and
a member of the working party that developed the ISTQB Foundation Syllabus. Dot
was awarded the European Excellence Award in Software Testing in 1999 and the first
ISTQB Excellence Award in 2012. Learn more about Dot at DorothyGraham.co.uk.

info@dorothygraham.co.uk
srttgmb@yahoo.com
© Seretta Gamba and Dorothy Graham 2014
www.DorothyGraham.co.uk
1
Test Automation Patterns
Prepared and presented by
© Seretta Gamba, Dorothy Graham 2014
Dorothy Graham
@DorothyGraham
2
Contents
Background
Issues and examples
Test automation Patterns Wiki
Conclusion

srttgmb@yahoo.com
3
Origin of patterns in software
– 1970s: architect Christopher Alexander describes
patterns for buildings (e.g. light from 2 walls)
– 1987: Kent Beck & Ward Cunningham
experimented & presented at OOPSLA
– 1994: “Gang of Four” wrote a book applying
pattern principles to software design
•  “Design Patterns: elements of reusable object-oriented
software”, Erich Gamma, Richard Helm, Ralph Johnson,
John Vlissides
– now widely accepted in software development
community
4
What is a pattern?
– a generally reusable solution to a commonly
occurring problem within a given context
– a description or template for how to solve a
problem that can be used in many different
situations
– a pattern is NOT
•  a finished solution that can be ‘plugged in’ or
transformed directly into code
•  prescriptive
•  a set of steps
– a pattern needs to be implemented within context

srttgmb@yahoo.com
5
Where did these automation patterns
come from?
– Experiences book published 2012
– Seretta Gamba contributed Ch 21
•  Automation through the back door,
by supporting manual testing
– Seretta is a developer, used to
using patterns
– when she read the other chapters, she thought:
“Patterns! Patterns!”
– she wrote up test automation patterns as a book &
asked me to join her
– Word didn’t work very well, so we made a wiki
6
Introduction to the wiki
•  the Test Automation Patterns wiki
– ISSUES – describe problems in automation
– PATTERNS – proven ways to overcome problems
– other things: acknowledgements, references, some
failure patterns
•  in this presentation
– I am using an off-line version of the wiki for
demonstration purposes
– later I will tell you how you can gain access to the
“real” wiki

srttgmb@yahoo.com
7
Test automation patterns wiki
•  purpose: share information, ideas and
experiences about test automation, using
issues and patterns
•  mind-maps summarise issues and patterns
(not clickable as links)
•  issues and patterns are classified into sub-
sections (e.g. management, process, etc.)
•  “diagnostics” help you find your way into the
most relevant issues for you (new)
8
Contents
Background
Issues and examples
Conclusion

srttgmb@yahoo.com
9
Issues (problems)
•  what problems affect you in test automation?
– what causes the most trouble?
– what takes longer than it should?
– what is holding you back in your automation?
– what are you struggling with?
– what would you most like to improve?
10
Test automation Issues: 4 examples
•  HIGH MAINTENANCE COST
–  Maintenance of the test automation scripts takes a long
time and costs more than it’s worth.
•  NO PREVIOUS TEST AUTOMATION
–  Your first time automating, or automation has failed and
you are starting again.
•  STALLED AUTOMATION
–  Automation has been tried, but it never “got off the
ground” or has now fallen into disuse.
•  UNREALISTIC EXPECTATIONS
–  Management has unrealistic expectations about what
test automation can & can’t do.

srttgmb@yahoo.com
11
Issue: HIGH MAINTENANCE COST
•  Maintenance of the test automation scripts
takes a long time and costs more than it’s
worth
•  Questions to ask:
– what cost (or time) would be acceptable?
– how is the cost measured?
– Note: answers may be different in different
contexts
12
Issue: HIGH MAINTENANCE COST 1
•  What are your symptoms?
– When software changes, e.g. window sequence is
different, tests fail. More effort to edit the existing
tests than to start again. (Capture/replay used)
– Diagnostic question: Are the scripts structured and
reusable?
•  1) No, scripts are mainly “stand-alone” without much
reuse, automators are “re-inventing”
•  2) Yes, structured and reusable scripts, but it needs
automators to add new tests, and we are short of time
and/or automators
visit offline wiki 

srttgmb@yahoo.com
13
Issue: HIGH MAINTENANCE COST 2
•  What are your symptoms?
– No time allowed to maintain the automated tests
– Tests are written in the tool’s language, but the
tool has changed, or we have to change tools
14
HIGH MAINTENANCE COST (Tour) - 1
•  Symptom: SUT changes -> large cost to update
– MAINTAINABLE TESTWARE
– Symptom: Scripts are mainly “stand-alone” without
much reuse, automators are re-inventing
•  GOOD PROGRAMMING PRACTICES
–  DESIGN FOR REUSE, OBJECT MAP
– Symptom: Structured & reusable scripts,
automator / time shortage
•  DOMAIN-DRIVEN TESTING
–  ABSTRACTION LEVELS, KEYWORD-DRIVEN TESTING

srttgmb@yahoo.com
15
HIGH MAINTENANCE COST (Tour) - 2
•  Symptom: No time to maintain automated
tests (we know we should)
– MANAGEMENT SUPPORT
•  Explain technical debt, SELL THE BENEFITS
•  Symptom: Scripts in tool language, tool
changed or having to change tools
– TOOL INDEPENDENCE
– ABSTRACTION LEVELS
16
DOMAIN-DRIVEN TESTING
•  Uses a “Domain-specific test language”
– keywords from the application domain
•  insurance: New Policy or Make Claim
•  mobile: Make Call or Update Contacts
– each keyword is processed by a script that then
processes any data for that keyword
– tests in spreadsheet or databases
•  make it easy for business testers to add new tests
•  needs technical support (to set up the TESTWARE
ARCHITECTURE and when things go wrong)

srttgmb@yahoo.com
17
Testware architecture: abstraction levels
abstraction here:
easier to write
automated tests:
 widely used
DOMAIN-DRIVEN
TESTING
abstraction here: easier
to maintain, and change
tools:  long life
GOOD PROGRAMMING
PRACTICES,
GOOD DEVELOPMENT
PROCESS
testware
architecture

Testers

Test
Execu/on
Tool

runs
scripts

High Level Keywords
Structured Scripts
structured

testware

Test
Automator(s)
write
tests
(in
DSTL)

18
Some other patterns
•  FAIL GRACEFULLY
– make sure a test “cleans up” after itself even if it
fails (so subsequent tests can run without worry)
– alternative: FRESH SETUP, where each tests re-
sets its initial state, no matter what happened
before, or SHARED SETUP
•  KILL THE ZOMBIES
– make sure tests are actually doing something;
delete or archive tests no longer in use

srttgmb@yahoo.com
19
Contents
Background
Issues and examples
Conclusion
20
About these patterns (and the wiki)
•  the wiki has a limited set of patterns
– and a limited “depth” for each pattern
•  patterns use other patterns
– it can get rather complex
•  we are focusing on system-level automation
rather than unit test automation
– See Gerard Meszaros’ book “xUnit Test Patterns:
Refactoring test code” for unit test patterns
•  these are test automation patterns, not test
patterns

srttgmb@yahoo.com
21
Diagnostics (new)
•  modeled on a doctor’s questions to a new
patient – ask questions, and ask more
questions, depending on the answers
•  First level
– new to automation? Go to NO PREVIOUS TEST
AUTOMATION
– completely happy? giving up (good riddance)?
•  share your story
– want to improve or revive your automation?
•  see next level
22
Improve or revive automation
•  Most pressing issue:
– lack of support (from mgt, testers, dev)
– lack of resources (staff, s/w, h/w, time)
– lack of direction (what to aut, what architecture)
– lack of specific knowledge (of SUT, tool,
maintainability)
– unsatisfactory quality of automation (ROI,
maintenance, unreliable)
•  each takes you to more detail in the wiki, and
points you to an issue with resolving patterns

srttgmb@yahoo.com
23
Lack of support
•  different kinds of support may be lacking
– managers pay “lip service” – don’t see the value
– managers have UNREALISTIC EXPECTATIONS
– testers don’t help the automators
– developers don’t help the automators
– specialists don’t help the automators (DB,
networks)
– no one helps the newbies
24
Contents
Background
Issues and examples
Conclusion

srttgmb@yahoo.com
25
Conclusion
•  we hope our wiki will help you identify and
understand problems (issues) and give you
ideas (patterns) for dealing with them
•  the wiki is free, by invitation
– email me at info@DorothyGraham.co.uk for an
invitation
– if you already have a wikispaces account, also give
me your wiki user name
– an invitation will be sent from the wiki giving you
instructions for how to set up your user name &
password
26
Finally
•  Thank you for coming today
•  Any questions?
•  Contact us:
– info@dorothygraham.co.uk
– srttgmb@yahoo.com
•  We welcome your feedback 
•  We welcome your stories & experiences
•  All the best with your test automation!

Test Automation Patterns

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (16)

Semelhante a Test Automation Patterns

Semelhante a Test Automation Patterns (20)

Mais de TechWell

Mais de TechWell (20)

Último

Último (20)

Test Automation Patterns