Presented in World Conference - Next Generation Testing, Bangalore on 11th July 2013

1. Building
the
“right”
regression
suite
using
Behavior
Driven
Tes5ng
(BDT)
Anand
Bagmar
So#ware
Quality
Evangelist
Principal
Consultant,

2. abagmar@gmail.com
Anand.Bagmar@thoughtworks.com
@BagmarAnand
about.me/anand.bagmar
About
me

3. What
do
you
expect
from
this
session?

4. Case
Study
• MulHple,
long
running
projects
• Legacy
applicaHons
• Integrated

5. • Limited
/
flaky
/
long-­‐running
AutomaHon
• Long
regression
cycle
• Huge
cost
of
fixing
defects
FuncHonal
TesHng
Challenges

6. • BriRle
• LiRle
/
less
value
• No
visibility
into
what
is
tested
• Maintenance
nightmare
• Outdated
• Expensive
As
a
result

7. Biggest
problem
• Trust
deficit
on
the
team
– In
exisHng
AutomaHon
– In
manual
tesHng
• Finger
poinHng
/
blame
game

8. My
Case
Study

10. Concept
#
1
The
Test
Pyramid

11. AutomaHon
–
A
Safety
Net

12. • “Succeeding
with
Agile”
–
Mike
Cohn
• MarHn
Fowler
–
Test
Pyramid
– hRp://marHnfowler.com/bliki/TestPyramid.html
The
Test
Pyramid

13. Func%onal
Tests
(GUI)
Web
Service
Tests
View
Tests
JavaScript
Tests
Integra%on
tests
Component
tests
Unit
Tests
Test
AutomaHon
Pyramid
Manual
/
Exploratory
TesHng
Product
under
test
Ideal
Test
Pyramid
Business-­‐
facing
Tests
Technology-­‐
facing
Tests
• Cost
• Effort
• Time

15. Test
Pyramid
reality
Development
team
Test
Pyramid
QA
team
Test
Pyramid

16. Func%onal
Tests
(GUI)
Web
Service
Tests
…
…
Unit
Tests
Manual
/
Exploratory
TesHng
Test
Ice-­‐cream
cone
Business-­‐
facing
Tests
Technology-­‐
facing
Tests
Test
Pyramid
–
AnH
PaRern
Product
under
test

17. Func%onal
Tests
(GUI)
Web
Service
Tests
…
…
Unit
Tests
Manual
/
Exploratory
TesHng
Product
under
test
Ideal
Test
Pyramid
Business-­‐
facing
Tests
Technology-­‐
facing
Tests
IdenHfy
the
right
type
of
tests
that
sit
on
top
of
the
Test
Pyramid

18. Concept
#
2
BDD
&
BDT

19. Behavior
Driven
Development
(BDD)

20. RED
GREEN
REFACTOR

21. Behavior
Driven
TesHng
(BDT)

22. The
Big
Picture
• How
is
this
funcHonality
going
to
be
used
by
the
end-­‐user?
• What
is
their
thought
process
going
to
be
when
using
this
product?
• What
is
the
“core-­‐business-­‐value”
I
am
delivering
to
the
end-­‐user?

23. Concept
#
3
Test
Specifica5on
Styles

24. Test
specificaHon
styles
• ImperaHve
• DeclaraHve

25. Search
Search
Results
Contact
Informa%on
An
example
For
SomeAwesomeAirlines
website
–
A
Guest
User
can
…

26. ImperaHve
style
Given
I
am
a
guest
user
on
the
home
page
And
I
choose
“round”
trip
opHon
And
I
select
“Chicago”
from
the
origin
dropdown
And
I
select
“San
Francisco”
from
the
desHnaHon
drop
down
And
I
select
departure
date
as
“5
December
2013”
And
I
select
returning
date
as
“25
December
2013”
When
I
click
on
Search
Then
I
should
see
the
search
results
page
And
I
should
see
at
least
1
opHon
for
my
criteria
...

27. …
When
I
select
the
“first”
opHon
Then
I
am
on
the
Contact
InformaHon
page
When
I
enter
first
name
as
“foo”
And
I
enter
last
name
as
“bar”
…
…
And
I
click
the
“Select
and
Con%nue”
buRon
Then
I
should
be
on
the
next
page

28. Given
I
am
a
guest
user
When
I
search
for
flight
opHons
for
a
“one-­‐way”
trip
for
“1”
“Adult”
from
“Chicago”
to
“San
Francisco”
And
I
select
the
“first”
flight
And
I
enter
“valid”
contact
details
for
“traveller1”
Then
I
am
able
to
Save
and
ConHnue
DeclaraHve
style

29. References
• Ben
Mabey
-­‐
hRp://benmabey.com/2008/05/19/
imperaHve-­‐vs-­‐declaraHve-­‐scenarios-­‐in-­‐user-­‐
stories.html
• Alister
ScoR
-­‐
hRp://waHrmelon.com/2010/12/10/
cucumber-­‐imperaHve-­‐or-­‐declaraHve-­‐that-­‐is-­‐
the-­‐quesHon/

30. My
Case
Study

34. • Remove
the
ambiguity
• Same
visibility
for
all

35. How
did
we
accomplish
this?

37. • IdenHfy
business
flows
based
on
personas,
and
how
the
personas
use
the
business
funcHonality,
or,
• IdenHfy
business
flows,
and
how
different
user
personas
can
use
the
business
flows

39. 1st
candidate
• Automate!
• High
Value,
Low
Cost
2nd
candidate
• Automate!
• High
Value,
High
Cost
Can
Automate?
• Manual
Regression?
• Low
Value,
Low
Cost
Do
Not
Automate
• Manual
Regression!
• Low
Value,
High
Cost
High
Value
High
Cost
Low
Value
Low
Cost

40. Who
is
doing
the
acHon?
What
is
the
business
funcHonality?
Depicts
a
user
flow
/
user
journey
Given
auc%oneer
creates
an
aucHon
And
5
vehicles
are
added
to
the
aucHon
When
auc%oneer
starts
the
aucHon
And
auc%oneer
sets
a
starHng
bid
of
5000
And
buyer_1
bids
And
buyer_2
bids
Then
auc%oneer
Sells
the
vehicle
And
buyer_2
wins
the
vehicle

41. What
next?
• IdenHficaHon
of
Tests
• Automate
it!

42. Automate
tests
that
are
valuable,
not
because
you
can
or
like
to
automate!

43. Who
is
doing
the
acHon?
What
is
the
business
funcHonality?
Depicts
a
user
flow
/
user
journey
Ac5ons
and
verifica5ons
happen
implicitly
Given
auc%oneer
creates
an
aucHon
And
5
vehicles
are
added
to
the
aucHon
When
auc%oneer
starts
the
aucHon
And
auc%oneer
sets
a
starHng
bid
of
5000
And
buyer_1
bids
And
buyer_2
bids
Then
auc%oneer
Sells
the
vehicle
And
buyer_2
wins
the
vehicle

44. Delete
profile

45. 1st
candidate
• Automate!
• High
Value,
Low
Cost
2nd
candidate
• Automate!
• High
Value,
High
Cost
Can
Automate?
• Manual
Regression?
• Low
Value,
Low
Cost
Do
Not
Automate
• Manual
Regression!
• Low
Value,
High
Cost
High
Value
High
Cost
Low
Value
Low
Cost

46. Final
state
Project
Management
tool
Mingle
Test
case
repository
(manual
+
automated)
Cucumber
.feature
files
with
appropriate
tagging
Feature
files
/
Scenarios
33
/
65
#
of
Manual
Tests
0
Tests
running
on
CI
(Jenkins)
Since
the
1st
test
was
automated
Smoke
tests
execuHon
Hme
15
minutes
Regression
tests
execuHon
Hme
45
minutes

47. How
did
this
technique
ever
become
successful?

48. Context
is
KING!
Domain
Product-­‐
under-­‐test
Distributed
Teams
Skills
Capability
Tech
Stack
Process
Tools

49. BDT
will
….
• NOT
work
in
isola5on

50. Requires
more
dedicated
and
focused
tesHng
(manual
/
exploratory)

51. Requires
lower
layers
of
Test
Pyramid
to
be
automaHon
and
implemented
well!

52. Func%onal
Tests
(GUI)
Web
Service
Tests
…
…
Unit
Tests
Manual
/
Exploratory
TesHng
Product
under
test
Ideal
Test
Pyramid
Business-­‐
facing
Tests
Technology-­‐
facing
Tests
IdenHfy
the
right
type
of
tests
that
sit
on
top
of
the
Test
Pyramid

53. AddiHonally
…
• Builds
and
deployments
are
automated
• FuncHonal
Performance
and
Concurrent
TesHng
automated
and
run
on
demand

54. BDT
Value
ProposiHon

55. ?
Incrementally
build
User
Flows
Regress
required
Product
Business
Value
Tests
in-­‐sync
Effec%ve
Tests
Remove
ambiguity
Understand
new
func%onality
Living
Documenta%on
Onboarding

56. Executable
Specs
BDT
Scenarios

58. Func%onal
Tests
(GUI)
Web
Service
Tests
…
…
Unit
Tests
BDT
helps
iden5fy
the
“right”
type
of
regression
tests!
Manual
/
Exploratory
TesHng
Product
under
test
Ideal
Test
Pyramid

59. Tools
to
enable
BDT
• Whiteboards
• Mind
maps
• Flow
charts
• Collabora5on!!

60. Automate
the
top
layer
Automate
using
-­‐
• BDD
Tools,
or,
• Directly
in
any
appropriate
programming
language
/
tools

61. abagmar@gmail.com
Anand.Bagmar@thoughtworks.com
@BagmarAnand
about.me/anand.bagmar
Thank
you