With the pressure of decreased time-to-market and the demand for error-free web applications, some people erroneously believe that test automation can solve all our defect woes. Dawn Jardine believes that neither test automation alone nor exploratory testing alone can solve quality issues. So, what is the best approach? Where do we invest our resources, time, and money so clients are happy with our product? How can automation and exploratory testing work together to obtain maximum test coverage? Join Dawn as she shares her experience of moving from a solely manual test case execution process to a combined automation and exploratory testing methodology. Learn how to apply critical thinking to automation and use creative energy during exploratory testing. Discover ways to modify your current test strategy to enable the QA team to deliver quality with confidence.
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf
Exploratory Testing and Automated Testing: Create a Healthy Relationship
1.
T24
Special
Topics
10/6/16
15:00
Exploratory
Testing
and
Automated
Testing:
Create
a
Healthy
Relationship
Presented
by:
Dawn
Jardine
Razorfish
Brought
to
you
by:
350
Corporate
Way,
Suite
400,
Orange
Park,
FL
32073
888-‐-‐-‐268-‐-‐-‐8770
·∙·∙
904-‐-‐-‐278-‐-‐-‐0524
-‐
info@techwell.com
-‐
http://www.starwest.techwell.com/
2.
Dawn
Jardine
An
associate
quality
assurance
manager
for
Razorfish
in
Toronto,
Dawn
Jardine
has
worked
in
several
roles
in
her
career.
After
years
in
the
teaching
profession,
Dawn
changed
direction
and
entered
the
software
field
as
a
front-‐end
web
developer
for
an
eLearning
software
company.
Dawn
expanded
her
skills
and
expertise
to
include
instructional
design,
eventually
joining
the
QA
testing
team.
With
more
than
fifteen
years
of
experience,
she
has
participated
in
the
launch
of
several
eLearning
and
eCommerce
projects,
including
Canadian
Police
Knowledge
Network,
Canadian
Tire
Corporation,
Walmart
Canada,
and
American
Express
Membership
Rewards.
Passionate
about
quality
process
and
women
in
technology,
Dawn
can
be
found
on
Twitter
@dawnjardine.
6. 9/21/16
4
7
Automation can provide better coverage
• for a large code base
• when a live site is in maintenance mode
• when it supplements manual testing
8
Automation can find defects earlier
• finds defects introduced during sprint development
• supports process of continuous test-and-fix in lower life cycle
• reduces the cost of rework
7. 9/21/16
5
9
Automation speeds delivery to client
• continuous integration and delivery during sprint
• achieve 100% regression test case execution
• release small and release often
10
When do you
implement
automation?
8. 9/21/16
6
11
The decision to use automation
• current code base is growing
• addition of new features and enhancements are increasing
• completion time of manual test case execution is expanding
• releases are occurring frequently
12
What do
you automate?
9. 9/21/16
7
13
Candidates for automated tests
• repeated tests within sprint development cycles
• regression tests that are executed pre-release
• sanity tests that are being performed during releases to
production
14
How do you implement automation?
10. 9/21/16
8
15
Planning for automation
• choose a valuable platform
• involve your current QA team
• hire QA developer(s)
• keep the process agile
16
Exploratory Testing
and its Role
11. 9/21/16
9
17
Where does exploratory testing fit?
18
“Insanity is doing the same
thing over and over again
and expecting different
results”
~Albert Einstein
12. 9/21/16
10
19 James Bach, Exploratory Testing Explained
“An interactive process of
simultaneous learning,
test design, and test
execution.”
20
Does the QA team need specialized skills?
• experience is an asset but not a
requirement
• critical thinking and problem solving
• autonomy and independence
• creativity
13. 9/21/16
11
21
Can you “explore” if you already know the way?
• YES!
• it’s not User Acceptance testing
• there are many ways of navigating from Point A to Point B
22
The Mutual Relationship
14. 9/21/16
12
23
“Assumptions are the
termites of relationships”
~Henry Winkler
24
Build on the strengths of your team
• trust
• communication and collaboration
• empowerment
• confidence
15. 9/21/16
13
25
How We Make it Work
The challenges and successes
26
The story
• online retail e-commerce solution in production
• automation in place for pre-release regression
• an established agile process in development and delivery
• QA test coverage was still the bottleneck
16. 9/21/16
14
27
Management approval
for change
• respect and value of the QA team
• don’t be afraid to innovate!
• plan for smarter test coverage
28
Shifting the mindset
of the QA team
• information sessions about exploratory testing
• review of existing manual regression tests
• created user stories for high level scenarios
17. 9/21/16
15
29
Be creative, be organized and be brave!
• mind-mapping sessions for the high level scenarios
• manual and automation team participation
• absence of test steps
• traceability by recording the results
• organized a trial run for the regression cycle
30
Gain stakeholders
support
• business wants return on
investment
• transparency is key
• continually inform
• provide metrics
18. 9/21/16
16
31
Sample of metrics for status reports
Manual
Exploratory Tests
Desktop Browsers OS/Devices
Total
Desktop
Chrome
FF
IE11
MAC Safari
IOS8 Chrome
IOS9 Safari
IOS8 Safari
Android Chrome
#of Test Cases 91 9 15 10 12 14 9 12 10
Pass 91 9 15 10 12 14 9 12 10
Fail 0
Blocked 0 0 0 0 0 0 0 0 0
Unexecuted 0 0 0 0 0 0 0 0 0
Executed 91 9 15 10 12 14 9 12 10
Percentage ExecuHon
Complete 100% 100% 100% 100% 100% 100% 100% 100% 100%
32
Continued collaboration
• ongoing audits of current and
future release work
• review recorded results from
exploratory tests
• Repeat, Repeat, Repeat!
19. 9/21/16
17
33
Benefits of testing style
• the QA team’s confidence level grows
• the project management can stick to release timelines
• the business benefits by staying competitive
34
May 2014
• 261 manually executed
step-by-step test cases
• No automation in place
• 6 days x 4 QAs
January 2016
• 49 Exploratory Test cases
• 1140 total automated tests
covering 4 browsers
• 3 days x 2 QAs with focus
on Mobile
vs
Rewards & Results
20. 9/21/16
18
35
What’s next?
• automate the mobile web and mobile app regression tests
• streamline current exploratory tests
• change the format of our sprint functional tests
36
Conclusion
