TSQA: Something In The Way We Test

•

0 likes•10 views

This document discusses exploratory testing and compares traditional test case-driven approaches to more contemporary exploratory approaches. It notes that test cases are better thought of as outputs that convey relevant information rather than inputs used to direct testing. The document also emphasizes that exploratory testing is a verb describing the active process of doing testing rather than a noun referring to pre-defined tests. It advocates for change-focused testing and shares examples of moving from traditional test documentation like test case documents to more lightweight charters and specifications.

Software

@maaretp
Something in
the Way We Test
by Maaret Pyhäjärvi

@maaretp
Hiring a 15-yo as tester
“Find conversation starters”
“Document what you do exactly”

@maaretp
Even a broken clock is right twice a day.

@maaretp
Testing serves a
purpose: catch
information we
may be missing.

@maaretp
Testing
Contemporary
Exploratory
testing
The
Traditional
Way
The
Contemporary
Way
Write and
execute test
cases
Exploratory
testing
Test Automation

@maaretp
Cost of Documentation?
… writing
… reading
… maintaining

@maaretp
5000 test cases
My new project….
Validate that….
Verify that…
Shall be…
100% of team
changed

@maaretp
Test Case
Idea
Action
+
Documenting…

@maaretp
2003 insight:
- Architecture doc => abstract
- Test doc => concrete

@maaretp
Read-time Awareness
person reading for minute each
Eleven (11) work-days

@maaretp
A Smaller Sample
Gen 0: 66 shall-statements
Gen 1: 46 do-verify –statements, 12 manual
Gen II: 18 [feature] charters

@maaretp
Test cases are ideas not
actions and that is why you
cannot record them based
on the clicks you do.

@maaretp
Test cases are outputs from
testing, not inputs to it.

@maaretp
As outputs useful if:
They convey relevant
information without
extensive investment to read

@maaretp
Second Smaller Sample
Gen 0:
• 6 [feature] charters
• 4 feature files with 40 executable specs
• Developer intent captured in 209 unit
tests, 7 integration tests, 16 UI tests

@maaretp
Test case Charter Executable
Example
Ideas
(title only)

@maaretp
The Bigger Theme?
19 new tests
6 weeks overlap time remains
Change-focused testing
…a lot of work ahead of us

@maaretp
Exploratory
Testing the Verb
Doing Testing
INPUT OUTPUT

@maaretp
Input
Tester
Domain knowledge
Requirements and specifications
Testing knowledge
Miscellaneous knowledge
Doing
Testing
INPUT OUTPUT

@maaretp
Output
Better tester
Coverage
Information incl. defects and change requests
Documentation: Strategy
Documentation:Tests
Doing
Testing
INPUT OUTPUT

@maaretp
Exploratory
Testing the Noun
Organization
EXPECTATIONS OUTCOMES

@maaretp
2020 2016
MIATPP
Most Influential Agile Testing
Professional Person
#PayToSpeak #TechVoices
#EnsembleTesting #EnsembleProgramming #StrongStylePairing
#ExploratoryTesting #TestAutomation
#ModernAgile
#AwesomeTesters
Maaret Pyhäjärvi (from Finland)
Email: maaret@iki.fi
Twitter: @maaretp
Web: maaretp.com
Blog: visible-quality.blogspot.fi
(please connect with me through
Twitter or LinkedIn)
2019, 2020, 2021
https://exploratorytestingacademy.com
Ohjelmistotestaus ry
https://techvoices.org

Similar to TSQA: Something In The Way We Test

ATAGTR '20 Keynote: Building status

Maaret Pyhäjärvi

STARWest Workshop: Explore with Intent

Maaret Pyhäjärvi

AgileTD Scandia: Mob Testing

Maaret Pyhäjärvi

Exploratory Testing on Computer interfaces

Maaret Pyhäjärvi

For a decade of software product agile, we had worked in a structure where business responsibility of what to build was allocated to a product owner and the responsibility of how to build it was allocated to a development team. Product owner would maintain a backlog, act as voice of the customers. Until one day we realized that the choice of what to build or fix is hard, and critical to everyone’s success. If we wanted to do it poorly, we delegated it to a single product owner. We started a no product owner experiment. For three months, we experienced the development team delivering multitudes of value to what we had grown to expect, and innovate customer-oriented solutions in direct collaboration with customers. Team satisfaction and happiness bloomed. The experiment turned into a continuous way of working. Customer-focused team directly in touch with their customers performs better without a proxy. Join me to learn how the decision power shared for everyone in the team transformed the ability to deliver, and how collaboration is organized with product experts and business representatives. More details: https://confengine.com/agile-india-2019/proposal/7963/working-without-a-product-owner Conference link: https://2019.agileindia.org

Working without a Product Owner by Maaret Pyhajarvi at #AgileIndia2019

Agile Software Community of India

Agile India: Working without Product Owner

Maaret Pyhäjärvi

Tampere Testing Days: Exploratory Testing an API

Maaret Pyhäjärvi

Testbash Australia: Next Level Teamwork - Pairing and Mobbing

Maaret Pyhäjärvi

Agile2016: Exploratory Testing an API

Maaret Pyhäjärvi

MoT Sarajevo: How to get better at Exploratory testing

Maaret Pyhäjärvi

Talentum: Testiautomaation tekemisen taito

Maaret Pyhäjärvi

AADays Poland: Perfecting the Craft of Test Automation

Maaret Pyhäjärvi

Testing Portugal: Making Your Team Awesome as Just a tester

Maaret Pyhäjärvi

HUSTEF '21 Keynote: Hands Off Exploratory Testing - Managing at Scale

Maaret Pyhäjärvi

AppiumLite Keynote: Many Hats to Make a Tester

Maaret Pyhäjärvi

Many different roles contribute to building software: product owners, business specialists, testers. Yet knowledge of programming keeps these roles at a distance. In this webinar, Maaret will discuss a new powerful way of removing this distance for the benefit of our teams: mob programming. She’ll walk through her lessons: the pains of joining a mob as a non-programmer, and the gains of what a tester brings into a mob. This webinar serves as an inspiration to learning together, immersed in the experience of creating software as we’re learning. Diversity of thoughts and skill sets improved everybody and created a better product. view webinar recording - https://testhuddle.com/mob-testing-better-products-through-diversity/

Mob Testing: Better Products Through Diversity

TEST Huddle

Many different roles contribute to building software: product owners, business specialists, testers. Yet knowledge of programming keeps these roles at a distance. In this talk, we will discuss a new powerful way of removing this distance for the benefit of our teams: mob programming. We’ll walk through my lessons: the pains of joining a mob as a non-programmer, and the gains of what a tester brings into a mob. This talk serves as an inspiration to learning together, immersed in the experience of creating software as we’re learning. Diversity of thoughts and skillsets improved everybody and created a better product.

EuroSTAR Webinar - Mob testing

Maaret Pyhäjärvi

Data Science: A Mindset for Productivity Keynote at 2015 Ronin Labs West Coast CTO Summit https://www.eventjoy.com/e/west-coast-cto-summit-2015 Abstract Data science isn't just about using a collection of technologies and algorithms. Data science requires a mindset that solves problems at a higher level of abstraction. How do we model utility when we think about optimization? How do we decide which hypotheses to test? How do we allocate our scarce resources to make progress? There are no silver bullets. But I'll share what I've learned from a variety of contexts over the course of my work at Endeca, Google, and LinkedIn; and I hope you'll leave this talk with some practical wisdom you can apply to your next data science project.

Data Science: A Mindset for Productivity

Daniel Tunkelang

TestWorksConf: Exploratory Testing an API in Mob

Maaret Pyhäjärvi

Similar to TSQA: Something In The Way We Test (19)

ATAGTR '20 Keynote: Building status

STARWest Workshop: Explore with Intent

AgileTD Scandia: Mob Testing

Exploratory Testing on Computer interfaces

Working without a Product Owner by Maaret Pyhajarvi at #AgileIndia2019

Agile India: Working without Product Owner

Tampere Testing Days: Exploratory Testing an API

Testbash Australia: Next Level Teamwork - Pairing and Mobbing

Agile2016: Exploratory Testing an API

MoT Sarajevo: How to get better at Exploratory testing

Talentum: Testiautomaation tekemisen taito

AADays Poland: Perfecting the Craft of Test Automation

Testing Portugal: Making Your Team Awesome as Just a tester

HUSTEF '21 Keynote: Hands Off Exploratory Testing - Managing at Scale

AppiumLite Keynote: Many Hats to Make a Tester

Mob Testing: Better Products Through Diversity

EuroSTAR Webinar - Mob testing

Data Science: A Mindset for Productivity

TestWorksConf: Exploratory Testing an API in Mob

More from Maaret Pyhäjärvi

MoT Cork: Exploring Realities of Testing - From Theory to Practice

Maaret Pyhäjärvi

Testing Voices '21 keynote: Testing Becoming Harder To BeValuable

Maaret Pyhäjärvi

TestJSConf: Go Find What We May Have Missed

Maaret Pyhäjärvi

SauceCon: It's More Complicated than THAT

Maaret Pyhäjärvi

ATAGTR: Targeting Errors Of Omission

Maaret Pyhäjärvi

Pivotal Moments, Pivotal Experiences

Maaret Pyhäjärvi

Keynote at Sanae Beer.EX: The Automationist's Gambit

Maaret Pyhäjärvi

Testing Assembly: From Observations to Insights on Test Specialist Careers

Maaret Pyhäjärvi

Let's Do a Thing and Call it Foo

Maaret Pyhäjärvi

Selenium Conf: Patterns to Whole Team Test Automation Transformation

Maaret Pyhäjärvi

A new generation of testing is emerging, led by testers who believe test automation is an inseparable part of how we do exploratory testing. Yes, you read that right: exploratory testing with test automation, and on unit, api, ui and e2e levels all at our reach with team collaboration. We work on acceptance criteria and examples in our short increments, yet people are at their most creative on what could go wrong when they use the product or a piece of the product as their external imagination. When we’re creative, we need to do things we can’t do by hand, using programming to extend our reach. When we’re learning, we need to take notes now and to keep for later, again using programming to replenish information we deem worthwhile. We want to work on exploring a category of bugs before implementing, and a category of bugs before releasing, and a category of bugs after releasing. Last year I worked with a team that was transforming from a traditional manual test case + test automation -based approach to contemporary exploratory testing. This year I work with another team that takes this style of testing further than we’ve taken it before with whole-team testing. Let’s learn from a practitioner’s examples of ideas leading into this, changes enabling this, and results we are seeing with contemporary exploratory testing.

AxiomSummit 2022: A Practitioner's Guide to Contemporary Exploratory Testing

Maaret Pyhäjärvi

Geekle QA: Automationist's Gambit

Maaret Pyhäjärvi

Programming Assignments in Tester Interviews

Maaret Pyhäjärvi

While we draw parallels to Queen's Gambit the series on Netflix and Software Testing, this talk is not about how substance abuse is like testing. Instead, the movie guides us to to its name and figuring out what is this gambit. With exploratory testing, we have the opening moves, the midpart and the end game. The start can be anything but it builds our platform for the end game. And while exploratory testing, automation is part of that way of staying in tune with learning to win in the end game.

EuroSTAR '21 Keynote: Automationist's Gambit

Maaret Pyhäjärvi

Hands off Exploratory Testing - Managing Testing at Scale

Maaret Pyhäjärvi

Breakpoint 2021: Contemporary exploratorytesting

Maaret Pyhäjärvi

SLASSCOM: Breaking illusions with Testing

Maaret Pyhäjärvi

Selenium Summit 21: Contemporary Exploratory Testing

Maaret Pyhäjärvi

Mimmit Koodaa - Testiautomaatio on testauksen dokumentointia

Maaret Pyhäjärvi

Testing United - Social Software Testing Approaches

Maaret Pyhäjärvi

More from Maaret Pyhäjärvi (20)

MoT Cork: Exploring Realities of Testing - From Theory to Practice

Testing Voices '21 keynote: Testing Becoming Harder To BeValuable

TestJSConf: Go Find What We May Have Missed

SauceCon: It's More Complicated than THAT

ATAGTR: Targeting Errors Of Omission

Pivotal Moments, Pivotal Experiences

Keynote at Sanae Beer.EX: The Automationist's Gambit

Testing Assembly: From Observations to Insights on Test Specialist Careers

Let's Do a Thing and Call it Foo

Selenium Conf: Patterns to Whole Team Test Automation Transformation

AxiomSummit 2022: A Practitioner's Guide to Contemporary Exploratory Testing

Geekle QA: Automationist's Gambit

Programming Assignments in Tester Interviews

EuroSTAR '21 Keynote: Automationist's Gambit

Hands off Exploratory Testing - Managing Testing at Scale

Breakpoint 2021: Contemporary exploratorytesting

SLASSCOM: Breaking illusions with Testing

Selenium Summit 21: Contemporary Exploratory Testing

Mimmit Koodaa - Testiautomaatio on testauksen dokumentointia

Testing United - Social Software Testing Approaches

Recently uploaded

A Comprehensive Appium Guide for Hybrid App Automation Testing.pdf

kalichargn70th171

How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?

XfilesPro

This is a classic migration case study (the past, current and the future) at scale from a world-wide company transitioning from Confluent Platform and Confluent Cloud to self-managed Apache Kafka on Kubernetes using Strimzi. At Maersk, we have been architecting, designing and implementing our 3rd generation Event Streaming Platform. This platform is based on Kubernetes in Azure and using Strimzi to operate Apache Kafka at large scale, highly reliable, segregating data based on isolated use cases. Our 2nd generation was based on OnPrem Confluent Platform and Confluent Cloud and this presentation is the story of this migration and reasoning behind it. Furthermore, we would get into details on how we monitor (Grafana, Prometheus), alert (GoAlert and alert as code), operate and provide self-service solutions on top of Strimzi to enable business critical application in Maersk, implemented in GoLang using the GitOps deployment model with Flux and Kustomization among others. Finally, if time allows we will end with a demo of an open-source self service tool to monitor and explore the cluster with most wanted features such as topic message browsing and configuring and restarting connectors.

StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi

steffenkarlsson2

Scenarios are the central artifact of the Behaviour Driven Development (BDD) process. Although many teams use scenarios and tools like Cucumber or SpecFlow to automate them, in many cases their scenarios contain a lot of details, particularly test data, and therefore they become too complicated to support collaboration with the business. The "essential" principle of scenario writing (scenario formulation) states that only those details should be included in the scenario that are relevant for the outcome. This talk provides help for those who struggle implementing this principle or would be interested to learn how you can create brief and maintainable scenarios.

Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)

Gáspár Nagy

JustNaik Solution Deck (stage bus sector)

Max Lee

Agnieszka Andrzejewska - BIM School Course in Kraków

bim.edu.pl

APVP,apvp apvp High quality supplier safe spot transport, 98% purity

amy56318795

IT Software Development Resume, Vaibhav jha 2024

vaibhav130304

Microsoft 365 Copilot is available to Enterprise customers and smaller organizations from January 16, 2024. It goes for the price set at $30 per user/month across all purchasing channels and customer types. Microsoft 365 Copilot gives users experiences allowing for the composition and replying of emails, chats, documents, contacts, and meeting minutes.    Microsoft 365 Copilot is available across all sales channels including Enterprise Agreement and Cloud Solution Provider (CSP).  Also, for commercial customers with:   • Microsoft 365 E3  • Microsoft 365 E5  • Office 365 E3  • Office 365 E5  • Microsoft 365 Business Standard  • Microsoft 365 Business Premium  They will be able to purchase Copilot for Microsoft 365 add-on SKU across all Microsoft sales channels. See below the full list of the Microsoft sales channels for Copilot:  • Enterprise Agreement  • Enterprise Agreement Subscription  • Cloud Solution Provider (CSP)  • Microsoft Customer Agreement – Enterprise (MCA-E)  • Direct from Microsoft  • Education customers through Enrollment for Education Solutions  You can reach out to our team of Microsoft licensing experts concerning any on-premises or cloud related questions seeking for clarity. Send us an email…info@q-advise.com

Microsoft 365 Copilot; An AI tool changing the world of work _PDF.pdf

Q-Advise

Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...

Abortion Clinic

Migrating your customer service data from Gorgias to Re:amaze can greatly enhance your support operations, but it requires detailed planning and precise execution. "A Guideline to Gorgias to Re:amaze Data Migration" is crafted to help businesses manage this transition smoothly and efficiently. This comprehensive guide ensures that all aspects of the data migration process are covered, from initial planning to post-migration support.

A Guideline to Gorgias to to Re:amaze Data Migration

Help Desk Migration

INGKA DIGITAL: Linked Metadata by Design

Neo4j

OpenChain @ LF Japan Executive Briefing - May 2024

Shane Coughlan

Implementing KPIs and Right Metrics for Agile Delivery Teams.pdf

Victor Lopez

In his book, The Nature of the Physical World, Sir Arthur Eddington commented that “We have to appeal to the one outstanding law — the second law of thermodynamics — to put some sense into the world.” This sense-making goes beyond the physical world, too. Entropy is also essential in the fields of information and communication theory. During this lecture for the Princeton Plasma Physics Laboratory, lecturer Andrea Goulet discussed the application of entropy-related concepts in two communication systems: software and collaborative teams. She examined how concepts that help us understand systemic statistical disorder, such as ergodic systems, Lyapunov exponents, Kolmogorov-Sinai entropy, and Shannon-entropy can help us optimize for both software quality and innovation. She also provided several domain-specific models: Lehman’s Laws and Conway’s Law for software, as well as new models from her own research that relate to entropy and innovation. Entropy helps us understand the world and achieve great things. There is an underlying beauty in its principles that we can use to advance scientific discovery. When we understand the subtleties related to balancing surprise and structure, we increase our chances for effective collaboration and finding novel solutions to complex problems.

Entropy, Software Quality, and Innovation (presented at Princeton Plasma Phys...

Andrea Goulet

Data privacy is one of the most critical issues that businesses face. This presentation shares insights on the principles and best practices for ensuring the resilience and security of your workload. Drawing on a real-life project from the HR industry, the various challenges will be demonstrated: data protection, self-healing, business continuity, security, and transparency of data processing. This systematized approach allowed to create a secure AWS cloud infrastructure that not only met strict compliance rules but also exceeded the client's expectations.

Designing for Privacy in Amazon Web Services

KrzysztofKkol1

CompTIA Security+ (Study Notes) for cs.pdf

Furqanuddin10

The Impact of PLM Software on Fashion Production

Wave PLM

Skilrock is an igaming technology platform & lottery software provider to the Global Lottery & Gaming Industry with a strong presence across continents. As part of the $2.4 billion SUGAL & DAMANI GROUP, Skilrock leverages the group's extensive domain experience to offer the INFINITI gaming platform - a true Omni-Channel, Omni-Gaming Platform that can serve any lottery or gaming operator anywhere in the world. INFINITI serves Retail, iGaming & Self Service Channels with equal ease and at the same time supports a wide variety of games like Lotto, Keno, Bingo, Instant, Sports, Poker, Rummy, Casino and Slots. Our popular solutions are Scratch Lottery, Retail Lottery, Instant Lottery and other igaming and lottery solutions.

iGaming Platform & Lottery Solutions by Skilrock

Skilrock Technologies

Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...

rajkumar669520

Recently uploaded (20)

A Comprehensive Appium Guide for Hybrid App Automation Testing.pdf

How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?

StrimziCon 2024 - Transition to Apache Kafka on Kubernetes with Strimzi

Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)

JustNaik Solution Deck (stage bus sector)

Agnieszka Andrzejewska - BIM School Course in Kraków

APVP,apvp apvp High quality supplier safe spot transport, 98% purity

IT Software Development Resume, Vaibhav jha 2024

Microsoft 365 Copilot; An AI tool changing the world of work _PDF.pdf

Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...

A Guideline to Gorgias to to Re:amaze Data Migration

INGKA DIGITAL: Linked Metadata by Design

OpenChain @ LF Japan Executive Briefing - May 2024

Implementing KPIs and Right Metrics for Agile Delivery Teams.pdf

Entropy, Software Quality, and Innovation (presented at Princeton Plasma Phys...

Designing for Privacy in Amazon Web Services

CompTIA Security+ (Study Notes) for cs.pdf

The Impact of PLM Software on Fashion Production

iGaming Platform & Lottery Solutions by Skilrock

Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...

TSQA: Something In The Way We Test

1. @maaretp Something in the Way We Test by Maaret Pyhäjärvi

2. @maaretp Everyone can test

3. @maaretp Hiring a 15-yo as tester “Find conversation starters” “Document what you do exactly”

4. @maaretp Even a broken clock is right twice a day.

5. @maaretp Testing serves a purpose: catch information we may be missing.

6. @maaretp Testing Contemporary Exploratory testing The Traditional Way The Contemporary Way Write and execute test cases Exploratory testing Test Automation

7. @maaretp Cost of Documentation? … writing … reading … maintaining

8. @maaretp 5000 test cases My new project…. Validate that…. Verify that… Shall be… 100% of team changed

9. @maaretp Test Case Idea Action + Documenting…

10. @maaretp 2003 insight: - Architecture doc => abstract - Test doc => concrete

11. @maaretp Read-time Awareness person reading for minute each Eleven (11) work-days

12. @maaretp A Smaller Sample Gen 0: 66 shall-statements Gen 1: 46 do-verify –statements, 12 manual Gen II: 18 [feature] charters

13. @maaretp Test cases are ideas not actions and that is why you cannot record them based on the clicks you do.

14. @maaretp Test cases are outputs from testing, not inputs to it.

15. @maaretp As outputs useful if: They convey relevant information without extensive investment to read

16. @maaretp Second Smaller Sample Gen 0: • 6 [feature] charters • 4 feature files with 40 executable specs • Developer intent captured in 209 unit tests, 7 integration tests, 16 UI tests

17. @maaretp Test case Charter Executable Example Ideas (title only)

18. @maaretp The Bigger Theme? 19 new tests 6 weeks overlap time remains Change-focused testing …a lot of work ahead of us

19. @maaretp Exploratory Testing the Verb Doing Testing INPUT OUTPUT

20. @maaretp Input Tester Domain knowledge Requirements and specifications Testing knowledge Miscellaneous knowledge Doing Testing INPUT OUTPUT

21. @maaretp Output Better tester Coverage Information incl. defects and change requests Documentation: Strategy Documentation:Tests Doing Testing INPUT OUTPUT

22. @maaretp Exploratory Testing the Noun Organization EXPECTATIONS OUTCOMES

23. @maaretp Testing Contemporary Exploratory testing The Traditional Way The Contemporary Way Write and execute test cases Exploratory testing Test Automation

24. @maaretp 2020 2016 MIATPP Most Influential Agile Testing Professional Person #PayToSpeak #TechVoices #EnsembleTesting #EnsembleProgramming #StrongStylePairing #ExploratoryTesting #TestAutomation #ModernAgile #AwesomeTesters Maaret Pyhäjärvi (from Finland) Email: maaret@iki.fi Twitter: @maaretp Web: maaretp.com Blog: visible-quality.blogspot.fi (please connect with me through Twitter or LinkedIn) 2019, 2020, 2021 https://exploratorytestingacademy.com Ohjelmistotestaus ry https://techvoices.org

TSQA: Something In The Way We Test

Recommended

Recommended

More Related Content

Similar to TSQA: Something In The Way We Test

Similar to TSQA: Something In The Way We Test (19)

More from Maaret Pyhäjärvi

More from Maaret Pyhäjärvi (20)

Recently uploaded

Recently uploaded (20)

TSQA: Something In The Way We Test