1. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
A Tester’s Role
on an
Agile Project
Agile Tester Days
Belgium
September, 2009
Janet Gregory, DragonFire Inc.
With material from Lisa Crispin
1
2. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Introduction - Me
First agile team – 2000
Currently – coaching, testing
Agile Testing: A Practical Guide for Testers
and Agile Teams; Addison Wesley 2009
2
Copyright 2009: Janet Gregory, DragonFire
3. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Topics
What about you !!!!
A very little bit about agile
Challenges
What is agile testing
Life of a tester during an iteration
Why and how testing must “keep up”
Continual improvement
3
Copyright 2009: Janet Gregory, DragonFire
4. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Agile is a term to describe methodologies that:
have short iterations
encourage active customer participation
demand whole team collaboration
test features as they are coded
deliver business value at regular intervals
adapt their processes based on feedback
…and so on………….
5. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Blurred Lines Between Roles
Agile developers are “test infected”
Agile testers and programmers collaborate
Agile testers and customers collaborate
“Whole Team” responsibility for testing
Everyone understands the business
5
5
Copyright 2009: Janet Gregory, DragonFire
6. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
What challenges do you see for
agile testers?
6
Copyright 2009: Janet Gregory, DragonFire
7. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Exercise
Break into groups
Discuss your current project or previous
projects
What type of problems is your team
experiencing?
Choose the top 3 problems you would like
to solve
7
Copyright 2009 Lisa Crispin and Janet Gregory
8. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Some Common Challenges
Separate test team
No common language
Fear – loss of identity
Testing always at the end
Organizational culture
Don’t know enough about agile concepts
Lack technical and tool skills
Physical limitations – work space
8
Copyright 2009: Janet Gregory, DragonFire
9. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
What is Agile Testing?
Testers are part of the team
Drives development with tests
Work with customers to define acceptance
tests for each story / feature
Provides continuous feedback to the team
Provides constructive skepticism
Tests each story as it is complete
9
Copyright 2009: Janet Gregory, DragonFire
10. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Management Expectations
Phased approach… 90% done… until it’s too late
Agile – new expectations
Make quality the priority (not speed!)
Let the team solve problems
Give team time to learn / slack time
Tips / Techniques
Provide visibility – raise risks early
Stress adaptive rather than predictive
Stress ‘getting business value’ regularly
Build credibility / trust by proving it works
10
Copyright 2007: Lisa Crispin and Janet Gregory
11. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
The agile tester’s mindset .....
Instead of
– We’re here to break the software!
Think
– What can we do to help deliver
the software successfully?
11
12. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Be involved, question, be proactive
13. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Whole Team Approach
Anyone can pick up a testing task
Run automated regression tests
regularly
Continuous build
Minimally – daily!
Otherwise ……….
13
Copyright 2009: Janet Gregory, DragonFire
14. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Discuss ....
Instead of saying NO,
or being the gatekeeper
Be the information provider so
business can make the decisions
14
Copyright 2009: Janet Gregory, DragonFire
15. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Change how we think about testing …
Focus on the 'why' we test
Change the way we think
not about the when
not about the how
white box, black box
15
Copyright 2009 Janet Gregory, DragonFire
16. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
The Agile Testing Quadrants
Brian Marick introduced the concept
Can be used to ensure we accomplish all goals
Q1 - Technology-facing tests that support the team
Q2 - Business-facing tests that support the team
Q3 - Business-facing tests that critique the product
Q4 - Technology-facing tests that critique the product
16
Copyright 2009 Janet Gregory, DragonFire
17. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Test Approach - The Agile Way
Project Initiation Get an understanding of the project
Release Planning Participate in estimating stories Create Test Plan
Each Iteration Write and execute story tests
1 Write and execute new functional test cases
…. Pair test with other testers, developers
X Automate new functional test cases
Run automated regression test cases
Perform Load Test
System Test / Complete Regression Test
End Game Perform UAT
Perform Mock Deploy
Participate in Release Readiness
Release to Prod/
Support Participate in Release to Prod
Participate in Retrospectives 17
Copyright 2008 Janet Gregory, DragonFire
18. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Project Initiation
Purpose – high level understanding of project
Questions to ask:
Are other teams involved?
Will we need special resources?
Test environments, tools, people
What other risks are there?
May be asked for 10,000 ft estimates
Required to determine cost/benefit
18
Copyright 2008 Janet Gregory, DragonFire
19. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Release Planning – The Purpose
Understand the “Conditions of Satisfaction”
Criteria to gauge success of the project
Functionality, quality, time???
Want to get a good idea of relative ‘size’
Plan what might be possible
19
Copyright 2008 Janet Gregory, DragonFire
20. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
What do you think a tester’s role is
during release planning ....
If anything?
20
Copyright 2009: Janet Gregory, DragonFire
21. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Release Planning – Tester’s Role
Sizing of stories
think about impacts to system
Ask questions
that may impact the ‘bigness’ of the story
to uncover hidden assumptions
such as ‘what if?’, or “what happens if?”
Not the time for details
Create a project “test plan”
21
Copyright 2008 Janet Gregory, DragonFire
22. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Preparing for Iteration - Stories
Be proactive - preplanning
Specifications workshops – Gojko Adzic
Help customer achieve ‘advance clarity’
on stories
Customers speak with “one voice”
Testable stories
Steel threads
22
Copyright 2009: Janet Gregory, DragonFire
23. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Use Pictures to Clarify
23
Copyright 2009: Janet Gregory, DragonFire
24. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Learning to Collaborate
Groups again....
Step 1 – Draw your vision of the perfect ....
Step 2 – Collaborate to create one common
vision that each of you can live with.
24
Copyright 2009 Lisa Crispin and Janet Gregory
25. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
During Iteration Planning
• Find hidden assumptions – question assumptions
• Ask “why?” Ask “what if?”
• Define and estimate testing tasks
• Ask what other tasks are required
• Define high level story tests or examples
25
Copyright 2009: Janet Gregory, DragonFire
26. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Testable Stories
Goal – steel thread, thin slice, critical path
Work towards an end-to-end first cut
What is absolutely required
Keep it simple
Allows whole team to work together
Gives good basis for testing
Later – add complexity / extras
26
Copyright 2008 Janet Gregory, DragonFire
27. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Drive Development - ATDD
ATDD (Acceptance Test Driven Development)
Express the intent of the story
Use examples
Defines behaviours
Maybe misbehaviours
Help the developers to understand what is
expected
27
Copyright 2008 Janet Gregory, DragonFire
28. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Acceptance Test Exercise
The story…..
As an on-line shopper, I want to provide my
shipping address when I check out so that my
order goes to the right location.
Consider
Locations – countries?
Misbehaviours?
What is the simplest thing we could do?
28
Copyright 2008 Janet Gregory, DragonFire
29. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
When Coding Starts
Write detailed test cases from examples
Make them executable if possible
Write simple, happy path tests to automate
Share them with the programmers
Once simple tests pass, write more test cases
Start with happy path
Then move towards edge cases
Identify possible exploratory scenarios
29
Copyright 2009: Janet Gregory, DragonFire
30. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Testing/Coding: Don’t sit and wait!
Is any testable part of a story ready?
Test with behind-the-GUI tool such as FIT?
Or other harness to bypass GUI
Pair with programmers
Test together before check-in
Show them issues
Ask questions
Bugs found here are cheap and easy to fix
30
Copyright 2009: Janet Gregory, DragonFire
31. F -X C h a n ge F -X C h a n ge
PD PD
!
!
W
W
O
O
N
N
y
y
bu
bu
to
to
k
k
lic
lic
C
C
w
w
m
m
w w
w
w
o
o
.d o .c .d o .c
c u -tr a c k c u -tr a c k
Testing/Coding: Collaboration
Pair with customers
Show customer early
Does delivered feature meet expectation?
Other team members, e.g.
Work with DBA to understand database
Work with sys admin to get better messaging
from continual build
31
Copyright 2009: Janet Gregory, DragonFire