Exploratory testing is an approach to testing that is often misunderstood. Actually it is an approach to testing that emphasizes testers ability to explore an unknown object or area through concurrent test design and test execution. In order to be skillful in it, you need to have several assets that will help you reveal relevant information about the product you’re building. I will share many of the things that I consider essential for a person who wants to understand more about exploratory testing. Talk is focusing a lot on things that have evolved my thinking and therefore helped me be more valuable as a tester. Many of them are also useful for anyone working in software development.
8. EXPLORATORY
TESTING
An approach to testing that emphasizes testers
ability to explore an unknown object or space
through concurrent test design and test execution.
Explore = travel through (an unfamiliar area) in order to
learn about it (Oxford Dictionaries)
@al3ksis
9. FREEDOM &
RESPONSIBILITY
“A style of software testing that emphasizes the personal
freedom and responsibility of the individual tester
to optimize the quality of his or her work by treating
test design, test execution, test interpretation, and test-related
learning as mutually supportive activities that continue in parallel
throughout the project.” – Cem Kaner
@al3ksis
10. CREATING BY
REACTING
“An essential aspect of a painter's canvas and a
musical instrument is the immediacy with which
the artist gets something there to react to.
A canvas or sketchbook serves as an "external
imagination", where an artist can grow an idea
from birth to maturity by continuously reacting to
what's in front of him.” – Bret Victor (Learnable
Programming)
@al3ksis
12. 1. QUALITY IS A
RELATIONSHIP
“Quality is value to some person.” – Jerry Weinberg
“…who matter.” – James Bach & Michael Bolton
By “value,” I mean, “What are people willing to pay
(do) to have their requirements met.” – Jerry
Weinberg
@al3ksis
13. "You can't separate the quality of
your employees work-life from
the quality of your products.”
(Source: W. Edward Deming Quality
Guruhttps://www.youtube.com/watch?v=YQpY3lnljBE - 5:22)
@al3ksis
31. WHAT DO WE SAY WHEN
WE SEE A CURVE LIKE
THIS?
@al3ksis
32. SUMMARY
• Exploratory Testing:
• Mental Engagement, Responsibility & Emphasizing
Exploration
• Quality:
• Quality is value to some person(s). Who is the person(s) that
matter?
• Communication Skill
• Check Intake, Check Meaning
• “What observation led you to that conclusion?”
• “What is your intention?” & “What are you trying to achieve
with what you said?”
• Leprechaun Hunter Mindset
• “Show Me the Data.”
@al3ksis
33. SOURCES OF
INFORMATION
• Evolving Understanding of Exploratory Testing (by
Michael Bolton) -
http://www.developsense.com/resources.html#exploratory
• Exploratory Testing (by Cem Kaner) -
http://www.testingeducation.org/BBST/exploratory/
• Telling Your Exploratory Story (by Jon Bach) -
http://www.youtube.com/watch?v=uUdWm7ddgfU
• Exploratory Testing (by Jon Bach) -
http://www.youtube.com/watch?v=8Y4WCdJRfV4
• A Case Against Test Cases (by Jon Bach) -
http://www.quardev.com/blog/a_case_against_test_cases
@al3ksis
34. SOURCES OF
INFORMATION
• The Leprechauns of Software Engineering (by Laurent
Bossavit) - https://leanpub.com/leprechauns
• Perfect Software: And Other Illusions about Testing (by
Jerry Weinberg) – You can find it from Amazon
• Explore It!: Reduce Risk and Increase Confidence with
Exploratory Testing (by Elisabeth Hendrickson) – You can
find it from Amazon
• Mindstorms: Children, Computers, and Powerful Ideas (by
Seymour Papert) – You can find it from Amazon
• Mindset: The New Psychology of Success (by Carol
Dweck) – You can find it from Amazon
• “The Idea Accelerator” (by Ken Hudson) – You can find it
from Amazon
@al3ksis
35. SOURCES OF
INFORMATION
• “What is Exploratory Testing?”
http://swtester.blogspot.ca/2012/05/what-is-exploratory-
testing.html (by Paul Carvalho)
• “The Little Black Book on Test Design”
http://thetesteye.com/blog/2011/09/the-little-black-book-on-
test-design/ (by Rikard Edgren)
• “To The New Tester”
http://www.satisfice.com/blog/archives/958 (by James Bach)
• “Fractal Exploratory Testing” http://www.a-sisyphean-
task.com/2013/01/fractal-exploratory-
testing.html#.UpxHGWRgY3Y (by Adam Knight)
• “Learnable Programming”
http://worrydream.com/LearnableProgramming/ (by Bret
Victor)
@al3ksis
36. SOURCES OF
INFORMATION
• “Two Forgotten Agile Values: Discipline and Skill”
http://www.exampler.com/discipline-and-skill.html (by Brian
Marick)
• “Practical Tools For Playing Well With Others”
http://oredev.org/2013/wed-fri-conference/practical-tools-for-
playing-well-with-others (by J.B. Rainsberger)
• “Patterns for The People” http://vimeo.com/19124478 (by
Kevlin Henney)
• “The Satir Interaction Model” http://sstc-
online.org/2006/pdfs/JB1365Notes.pdf (by Virginia Satir)
• “Test Heuristics Cheat Sheet” http://testobsessed.com/wp-
content/uploads/2011/04/testheuristicscheatsheetv1.pdf (by
Elisabeth Hendrickson)
@al3ksis
-Understanding about Exploratory Testing based on my understanding about Testing-Not universal definition but helps you to understand what I mean when I talk about testing
Could be “Experimenting with a product” – Didn’t want to restrict your thinking only on a concrete product (e.g. software)Testing can have different missions:Common one being “Find important bugs”Can be also to see if first build of a web site is testable. Yes experimenting with a product in order to evaluate, but not necessarily to find important bugs – instead see if the build is testable. Or will it crash from every click.
In the end testing is much about asking “What if?”.What if I log in with a user that has no access rights? What if I send a message with customer that has leading zeros on its customer number? What if I operate on web portal over 8 hours with less than 15 minutes breaks? I explain that sometimes these questions can lead to me asking a question from someone else, operating with the software by myself or perhaps with the help of some specific tool. All start from a thought that aims at learning more about the product and therefore revealing quality related information.
When we evaluate a test process, we need to evaluate the mental process of testing. Observing my actions will not tell you if I’m testing because thinking is crucial in testingWhat happens on the keyboard, or the questions we make, is important, but still a result of our thinking. I see similarities with how I’m playing with my son – for an outsider it might seem that I’m playing with him, but if I’m not mentally engaged, then I’m not playing with him.
Concentrate on the first part – open up the “concurrent test design & test execution” later with the help of a metaphorET is an approach, not a separate techniqueBy testers I mean the person that is testing, should not be confused to a role of tester.
If given freedom to think – need to honor that responsibilityExamples: not using tool when it is a good idea, not improving test data when it is a good idea
Product acts as a something there to react to“Concurrent test design and test execution” can be similar to continuously reacting to what’s in front of him.
-These helping me improve as a tester is my subjective opinion
Quality is a relationship that gets defined by people whose lives our products touch (Marc McNeill according to Dan North)Think of variety in what music, movies, clothes or food people Rationally picking whose value we satisfyOwn preferences affecting in how we develop softwarePeople willing to pay (do) to have their requirements met – might be users or ones building the softwareValue has it’s price
-More open minded with bugs – bug for who?
Communication skill is essential in software developmentEspecially because as a tester I need to learn as much as I can & communication skill is important in achieving that
Basically Satir Interaction Model – originally 7 steps but was reduced to 4 steps (not sure if it was by Virginia Satir)Different steps:Intake: words, but also -- facial expression, body position, skin color changes, scent, breathing, tone, paceMeaning: How do we interpret the intake?Significance: How we feel about the Meaning we have made of the Intake - How we feel about that feeling (OK? hurt? angry? joyous? confused?)Response
Intake: words, but also -- facial expression, body position, skin color changes, scent, breathing, tone, paceMeaning: How do we interpret the intake?Significance: How we feel about the Meaning we have made of the Intake - How we feel about that feeling (OK? hurt? angry? joyous? confused?)Response
Book about is basically about telephone game in software professionIt has good examples of how things like cost of defects and cone of uncertainty have surprisingly little credible research behind themPerhaps the most important take on from the book is the Leprechaun Hunter Mindset – strong curiosity to see behind the claims
I suspected that this was the source of the picture on Twitter
- Helmet had horsehair inside of it, to make the impact less painful
Claims come in many ways, they can be graphs or claims like theseWhat should be going on in your head is this: (next page)
If someone is trying to make a claim, I want to know what is it based on. After that the conversation can be more useful
