Behavior Driven Testing (BDT) is an evolved way of thinking about Testing. It helps in identifying the 'correct' scenarios, in form of user journeys, to build a good and effective (manual & automation) regression suite that validates the Business Goals.

1. BUILD THE “RIGHT”
REGRESSION SUITE USING
BEHAVIOR DRIVEN
TESTING - BDT
Anand Bagmar
Test Practice Lead

2. @BagmarAnand
about.me/anand.bagmar
ABOUT ME

3. Think ….

4. CASE STUDY
• Multiple, long running projects
• Legacy applications
• Integrated

5. • Limited / flaky / long-running Automation
• Long regression cycle
• Huge cost of fixing defects
FUNCTIONAL TESTING CHALLENGES

6. • Brittle
• Little / less value
• No visibility into what is tested
• Maintenance nightmare
• Outdated
• Expensive
AS A RESULT

7. • Trust deficit on the team
• In existing Automation
• In manual testing
• Finger pointing / blame game
BIGGEST PROBLEM

8. My Case Study

10. Concept #1
Given -> When -> Then

11. Expected Functionality:
The customer should be able to
withdraw money from his account via
an ATM machine

12. BDD Example:
Given the account has balance of 5000
When the customer requests 1000
Then the account is debited by 1000

13. 13

14. Workshop – Iteration 1
Requirements

16. PHASE 1 REQUIREMENTS
• Guest User is able to search for a flight for a
single traveller
• Select option from Search Results page
• Specify contact information for person
booking the flight

17. A Guest User can …
WORKFLOW
Search
Search
Results
Contact
Information

18. SEARCH MODULE

19. SEARCH RESULTS

20. CONTACT DETAILS

21. CONTACT DETAILS

22. PHASE 1 REQUIREMENTS
• Guest User is able to search for a flight for a
single traveller
• Select option from Search Results page
• Specify contact information for person
booking the flight

23. A Guest User can …
WORKFLOW
Search
Search
Results
Contact
Information

24. LETS GET STARTED … WRITE 2 TEST SCENARIOS

25. Iteration #1
Showcase

26. Concept #2
Test Specification Styles

27. • Imperative
• Declarative
TEST SPECIFICATION STYLES

28. A Guest User can …
AN EXAMPLE
Search
Search
Results
Contact
Information

29. IMPERATIVE STYLE
Given I am a guest user on the home page
And I choose “round” trip option
And I select “Chicago” from the origin dropdown
And I select “San Francisco” from the destination dropdown
And I select departure date as “5 December 2013”
And I select returning date as “25 December 2013”
When I click on Search
Then I should see the search results page
And I should see at least 1 option for my criteria
...

30. IMPERATIVE STYLE …
…
When I select the “first” option
Then I am on the Contact Information page
When I enter first name as “Anand”
And I enter last name as “Bagmar”
…
…
And I click the “Select and Continue” button
Then I should be on the next page

31. DECLARATIVE STYLE
Given I am a guest user
When I search for flight options for a “one-way” trip for “1”
“Adult” from “Chicago” to “San Francisco”
And I select the “first” flight
And I enter “valid” contact details for “traveller1”
Then I am able to Save and Continue

32. • Ben Mabey -
http://benmabey.com/2008/05/19/
imperative-vs-declarative-scenarios-in-
user-stories.html
• Alister Scott -
http://watirmelon.com/2010/12/10/
cucumber-imperative-or-declarative-that-
is-the-question/
REFERENCES

33. Workshop – Iteration 2
Evolved Requirements

34. • User registration from Contact information
page
• Registered user can login from home page
(below search criteria)
• For logged-in user, when booking ticket,
contact information is pre-filled with
logged-in user information
PHASE 2 REQUIREMENTS

35. Login Search
Search
Results
Contact
Information
• Pre-filled
WORKFLOW … ENHANCED

37. REGISTERED USER

39. • User registration from Contact information
page
• Registered user can login from home page
(below search criteria)
• For logged-in user, when booking ticket,
contact information is pre-filled with
logged-in user information
PHASE 2 REQUIREMENTS

40. Login Search
Search
Results
Contact
Information
• Pre-filled
WORKFLOW … ENHANCED

41. LETS GET STARTED … WRITE 2 TEST SCENARIOS

42. Iteration #2
Showcase

43. WHY IS THIS IMPORTANT?

44. Concept #3
The Test Pyramid

45. Test Automation
A PRACTICE THAT MAKES TEAMS SUCCESSFUL?

46. Test Automation
A PRACTICE THAT MAKES TEAMS UNSUCCESSFUL?

47. What is
Test Automation?

48. Automation
-
A Safety Net

49. Why do we do
Test Automation?

50. QUICK FEEDBACK

51. How to get quick
feedback from
Test Automation?

52. • “Succeeding with Agile” – Mike Cohn
• Martin Fowler – Test Pyramid
martinfowler.com/bliki/TestPyramid.html
THE TEST PYRAMID

53. IDEAL TEST AUTOMATION PYRAMID
Unit
Manual / Exploratory
Integration
JavaScript
View
Web Service
UI
Cost
Impact
Time
Business-
facing
Tests
Technology-
facing
Tests

54. REALITY

55. ICE-CREAM CONE ANTI PATTERN
Manual / Exploratory
UI
Web Service
View
JavaScript
Integration
Unit

56. Web Service
View
JavaScript
Integration
Unit
Developers Test Pyramid QA Team Test Pyramid
DUAL TEST PYRAMID ANTI-PATTERN
Manual / Exploratory
UI
Web Service

57. CUP-CAKE ANTI-PATTERN

58. Concept #4
Behavior Driven Testing
BDT

59. BEHAVIOR DRIVEN TESTING (BDT)

60. • How is this functionality going to be used
by the end-user?
• What is their thought process going to be
when using this product?
• What “core-business-value” should be
delivered to the end-user?
THE BIG PICTURE

61. Identify the “right” type of tests that sit
on top of the Test Pyramid

62. My Case Study

64. HOW TO SOLVE THE PROBLEM?

65. • Remove the ambiguity
• Same visibility for all
SOLUTION

66. • Whiteboards
• Mind maps
• Flow charts
• Collaboration!!
TOOLS TO ENABLE BDT

67. HOW DID WE ACCOMPLISH THIS?

68. IDENTIFY BUSINESS FLOWS

69. • Identify business flows based on personas,
and how the personas use the business
functionality, or,
• Identify business flows, and how different
user personas can use the business flows
NOTE: Business flows are different from UI
flows
HOW TO IDENTIFY BUSINESS FLOWS ?

70. PRIORITIZE BUSINESS FLOWS

71. 1st candidate
• Automate!
• High Value, Low Cost
2nd candidate
• Automate!
• High Value, High Cost
Can Automate?
• Manual Regression?
• Low Value, Low Cost
Do Not Automate
• Manual Regression!
• Low Value, High Cost
High
Value
High
Cost
Low
Value
Low
Cost

72. AUTOMATE THE TOP LAYER USING
• BDD Tools, or,
• Directly in any appropriate programming
language / tools

73. • Identification of Tests
• Automate it!
WHAT’S NEXT?

74. Caution!!

75. AUTOMATE TESTS THAT ARE VALUABLE,
NOT BECAUSE YOU CAN OR LIKE TO!
CAUTION!

76. Who is doing the action?
What is the business
functionality?
Depicts a user flow / user
journey
Actions & Verifications
happen implicitly
Given auctioneer creates an auction
And 5 vehicles are added to the
auction
When auctioneer starts the auction
And auctioneer sets a starting bid
of 5000
And buyer_1 bids
And buyer_2 bids
Then auctioneer Sells the vehicle
And buyer_2 wins the vehicle
EXAMPLE

77. Delete
profile
UPDATE BUSINESS FLOWS

78. 1st candidate
• Automate!
• High Value, Low Cost
2nd candidate
• Automate!
• High Value, High Cost
Can Automate?
• Manual Regression?
• Low Value, Low Cost
Do Not Automate
• Manual Regression!
• Low Value, High Cost
High
Value
High
Cost
Low
Value
Low
Cost

79. Project Management tool Mingle
Test case repository (manual
+ automated)
Cucumber .feature files with
appropriate tagging
Feature files / Scenarios 33 / 65
# of Manual Tests 0
Tests running on CI (Jenkins) Since the 1st test was
automated
Smoke tests execution time 15 minutes
Regression tests execution
time
45 minutes
FINAL STATE … IN MY CASE STUDY

80. • Builds and deployments were automated
• Tests ran on demand, automatically!
• Smoke / Regression / Javascript / Unit
• Functional Performance
• Concurrent Testing
ADDITIONALLY …

81. How did this technique
(BDT) become
successful?

82. CONTEXT IS KING!
Domain
Distributed
Teams
Skills
Capabilities Product-under test
Tech Stack
Process
Tools

83. BDT – Lessons Learnt

84. NOT work in isolation
BDT WILL …

85. BDT REQUIRES …
Lower layers of Test Pyramid to be well
automated!

86. BDT REQUIRES …
Dedicated and Focused testing (manual /
exploratory)

88. BDT helps identify the “right” type
of regression tests!

89. Executable
Specs
BDT
Scenarios
Business Idea
Features
Epics
Stories
Iterations
Releases
BDT VALUE PROPOSITION

90. BDT VALUE PROPOSITION …
Incrementally
build
Business Flows
Regress
Business Value
Test in-sync with
Product Functionality
Effective
Tests

91. BDT VALUE PROPOSITION …
Remove
ambiguity
Understand
new functionality
Living
Documentation
Onboarding

92. abagmar@gmail.com
Anand.Bagmar@thoughtworks.com
@BagmarAnand
about.me/anand.bagmar
THANK YOU