1. FROM STAKEHOLDER EXAMPLES TO LIVING DOCUMENTATION
Christian Hassa - ch@techtalk.at - @chr99ha
.NET User Group Bern, 5th of December 2012
Specification-By-Example
with Gherkin

2. TechTalk at a glance
• Agile consulting and delivery
• Offices in: Vienna, Budapest, Zurich
• Established in 1993
Vienna/Austria

3. 3
How do we specify
requirements?

4. 4
The purpose of user stories …
• Describe user needs
• Describe system features
• Unit of planning
• Reminder for a discussion
• Mechanism to defer a discussion

5. 5
Impact Mapping
Story Mapping
Specification-By-Example
Establishing a shared understanding
Why?
How?
Code
Acceptance
Criterion
Epic
Capability
Impact, Goal
Easier to define upfront Harder to define upfront
Reminder
for a
discussion
Bug
report
Isolated,
formalized
example
User Activity
User Story
Example

6. 6
Collecting Acceptance Criteria
“I would try to put a book into the
shopping cart …”
“I would try to remove a book
from the shopping cart…”
“I’d check whether the shopping cart
is empty, when I enter the shop …”
“I would try to add the same book
again to the shopping cart …”
Books can be placed into
shopping cart.
Books can be removed from
shopping cart.
Shopping cart should be empty
when entering the shop.
Adding same book again to
shopping cart should increase
quantity.
As a potential customer
I want to collect books in a shopping cart
So that I can order several books at once.
“Imagine this story is
already implemented:
how would you verify it?”

7. 7
Why
do we need
examples?

8. 8
UI wire frames,
existing UI
rules, key examples
existing artifacts,
samples
Examples for user stories

9. 10
Specification-by-Example
Examples …
• make abstract descriptions
better understandable
However …
• examples are usually not formally
exchanged or documented
Brian Marick
Examples Tests
Requirements
consist of
describe verify
fulfillment of

10. 11
Discussion of acceptance criteria
public void TestInitialOrderDiscount()
{
Customer newCustomer = new Customer();
Order newOrder = new Order(newCustomer);
newOrder.AddBook(
Catalog.Find(“ISBN-0955683610”)
);
Assert.Equals(33.75,
newOrder.Subtotal);
}
Register as “bart_bookworm”
Go to “/catalog/search”
Enter “ISBN-0955683610”
Click “Search”
Click “Add to Cart”
Click “View Cart”
Verify “Subtotal” is “$33.75”
We would like to encourage new users to
buy in our shop.
Therefore we offer 10% discount for their
first order.
Original idea for the illustration: George Dinwiddie
http://blog.gdinwidiee.com

11. 12
… illustrated with formalized examples
Given the user has not ordered yet
When the user adds a book with the price of EUR 37.5
into the shopping cart
Then the shopping cart sub-total is EUR 33.75.
Original idea for the illustration: George Dinwiddie
http://blog.gdinwidiee.com

12. 13
Discover hidden assumptions
Actually, this not quite right:
Books on sale should be excluded.
Original idea for the illustration: George Dinwiddie
http://blog.gdinwidiee.com

13. 14
Collaboration: 3 amigos
“Happy
Path”
Technical
feasability
Exceptions,
border cases
Original idea for the illustration: George Dinwiddie
http://blog.gdinwidiee.com

14. 15
Abstract acceptance criteria
As a shop visitor
I want to collect books in my shopping basket
so that I can purchase multiple books at once.
Books can be added to the shopping basket
Books can be removed from the shopping basket
Shopping basket is initially empty
The same book can be added multiple times to the shopping
basket

15. 16
Examples for acceptance criteria
As a shop visitor
I want to collect books in my shopping basket
so that I can purchase multiple books at once.
Books can be added to the shopping basket
Given my shopping basket is empty
When I add the book “Harry Potter” to my shopping basket
Then my shopping basket should contain 1 copy of “Harry Potter”

16. 17
As a shop visitor
I want to collect books in my shopping basket
so that I can purchase multiple books at once.
Books can be added to the shopping basket
Examples for acceptance criteria
When I add the book “Harry Potter” to my shopping basket
Then my shopping basket should contain 2 copies of “Harry Potter”
The same book can be added multiple times to the shopping basket
Given my shopping basket contains 1 copy of “Harry Potter”

17. 18
The same book can be added multiple times to the shopping basket
Structure of examples
Given my shopping basket contains 1 copy of “Harry Potter”
When I add the book “Harry Potter” to my shopping basket
Then my shopping basket should contain 2 copies of “Harry Potter”
Title: Describes intention/abstract acceptance criterion
Arrange: Context, initial state of the system
Act: Execution of the feature
Assert: Assertion of observable behaviour
And I should see the warning: “Book already existed in basket”
Triple-A
constraint
“Checks”
Chaining
up steps

18. 20
How long do we
care
about the
examples?

19. 21
Purpose of the examples
• Shared understanding:
acceptance criteria
• Documentation:
Look-up detail aspects
of the system
• Regression-tests:
Understand what
assumptions have been violated

20. 22
Continuous validation with automation
Given my shopping basket contains 1 copy of “Harry Potter”
When I add the book “Harry Potter” to my shopping basket
Then my shopping basket should contain 2 copies of “Harry Potter”
System
„Step Definitions“ are binding individual steps
to an automatable interface of the application.
Automatable
interface
UI
Automation
Automation does not necessarily have to bind to the UI.
Automatability of system is supported/evolving with development.

21. 23
Demo
http://www.specflow.org
Gherkin automation for .NET
• Visual Studio plugin (VS-Gallery)
• NuGet Package

22. 24
What does that
mean for
testing?

23. 25
UI automation becomes expensive
when …
• trying to automate
manual tests
• making tests
unreadable when
automating them
• automating after
completing
development
structure
readability
point in time

24. 26
Structure
Manual tests
Asserts Multiple combined
features
Structure ACT-ASSERT-
ACT-ASSERT-
ACT-ASSERT-
…
Dependent on other
features
Long test path with
high chance to break
Automated Check
Single aspect of a
single feature
ARRANGE –
ACT –
ASSERT
Independent of
other features
Short test path with
lower chance to break

25. 27
Test automation pyramid
User
journeys
Acceptance-
criteria
Units
exploratory
testing
Source: Mike Cohn
many
few
hard
easy
Automatability

26. 28
// Go to web page 'http://localhost:40001/' using new browser instance
BrowserWindow localhostBrowser = BrowserWindow.Launch(
new System.Uri(this.RecordedMethod1Params.Url));
// Click 'Register found item' link
Mouse.Click(uIFundstückerfassenHyperlink, new Point(56, 9));
// Click 'Save' button
Mouse.Click(uISpeichernButton, new Point(44, 14));
int fundNr1 = int.Parse(uIFundNr127Pane.InnerText.Substring(9));
// Click 'Register found item' link
Mouse.Click(uIFundstückerfassenHyperlink, new Point(63, 7));
// Click 'Save' button
Mouse.Click(uISpeichernButton, new Point(34, 11));
int fundNr2 = int.Parse(uIFundNr128Pane.InnerText.Substring(9));
Assert.IsTrue(fundNr1 + 1 == fundNr2);
// Click 'Close' button
Mouse.Click(uICloseButton, new Point(26, 11));
Readability

27. 29
A readable test case
Scenario: New found items should receive a
consecutive number for the current year
Given the previous found item of the
current year had the number 145
When I register a new found item
Then the last found item of the
current year should have the number 146

28. 30
When to test (point in time)
Acceptance criteria
(ATDD, BDD)
Unit Tests
(TDD)
business view
technical view
Exploratory tests
Workflow tests
Performance, Scalability,
Usability,Security, …
definingtheproduct
criticizingtheproduct
New dimension: defining the product
Synergy: Specification of requirements and tests
Agile Testing Quadrants: Brian Marick

29. 31
Cross-functional teams
Co-creation
Fast feedback

30. 32
Cross-functional work
Sprint 1 Sprint 2 Sprint 3
short iteration
US4
Plan
Implement &
autom. test
US5
Plan
Implement &
autom. test
US2
Plan
Implement &
autom. test
US3
Plan
Implement &
autom. test
US6
Plan
Implement &
autom. test
US1
Plan
Implement &
autom. test
US7
Plan
Implement &
autom. test
US8
Plan
Implement &
autom. test
US9
Plan
Implement &
autom. test
ExploratoryTests
Specification and test
Collaboration for defining
acceptance criteria
Collaboration for
automation
Preventing bugs
instead of finding them!
Extension of
“Test Cases”
Limit
WIP
Collaboration
in manual
testing

31. 33
Manual Testing is always necessary!
User
journeys
Acceptance-
criteria
Units
exploratory
testing
Source: Mike Cohn
many
few
harder
easier
Automatability
Manual
Check
after
Story
Done
Main
success
pathes
Undiscovered
acceptance
criteria
No/(few)
manual
regression
checks
Few pathes
are enough
More time
for exploration

32. 34
What does that
mean for
development?

33. 35
Workflow - TDD

34. 36
Extending TDD for business

35. 37
Transparency for stakeholders
In Progress

36. 38
Current sprint report: all sprint scenarios

37. 39
Starting with first scenario (AC)

38. 40
Finishing the first scenario (AC)

39. 41
Progressing scenario after scenario

40. 42
Progressing scenario after scenario

41. 43
Progressing scenario after scenario

42. 44
Implementing user stories in parallel

43. 45
First user story ready for testing

44. 46
Manual testing can start even earlier

45. 47
Already done work can break again

46. 48
Already done work can break again

47. 49
See what is temporarily not working

48. 51
Living
documentation

49. 52
Story Mapping
Optimizing and refining scope
Why?
How?
Code
Acceptance
Criterion
Epic
Capability
Impact, Goal
Easier to define upfront Harder to define upfront
User Activity
User Story
Example

50. 53
Story Maps
• Original concept by Jeff Patton
• Supports iterative product design
• Optimized for desired outcome or
deliverable the system should support

51. 56
Building story maps
Get mainstream book
fast and conveniently
delivered
Find book
I want
Collect
books
Complete
order
Wait for
book
Receive
book
time
browse
best
sellers
put into
basket
enter
address
receive
delivery
slip
receive
delivery
notificat.
pay with
credit
card
search
book by
title
create
wish list
inquiry
order
status
desired outcome
or deliverable
user
activities
system
features
necessity

52. 57
Walking
skeleton
Enabling build – measure - learn
Find book
I want
Collect
books
Complete
order
Wait for
book
Receive
book
time
browse
best
sellers
enter
address
receive
delivery
slip
pay with
credit
card
search
book by
title
create
wish list
inquiry
order
status
put into
basket
receive
delivery
notificat.
necessity
manual
workaround
omitted
steps
Get mainstream book
fast and conveniently
delivered

53. 60
Story Maps in ALM with TFS
TFS work items in story maps
http://speclog.net

54. 61
Story Map in SpecLog

55. 62
Sprint 1

56. 63
Sprint 2

57. 64
Sprint 3

58. 65
Sprint 4

59. 66
Dropped Scope

60. 67
Added Scope

61. 69
User Stories vs. Features
Product/Sprint Backlog
User Story 1
AccCrit 1
AccCrit 2
User Story 2
AccCrit 3
AccCrit 4
Living Documentation
Feature 1
AccCrit 1
AccCrit 2
Feature n
AccCrit 4
AccCrit m
User Story n
AccCrit 5
AccCrit m
AccCrit 3
AccCrit 5
„Done“
• Future options of the system
• Organized/refined according to
priority, value, effort, risk, ...
• Next possible increments of
the product (units of work)
• Current state of the system
• Organized/refined for
functional overview
• Versioned and maintained
together with source code

62. 71
Living documentation
• Link business readable automation from source code to story maps
• Feed execution results into story maps

63. 72
Additional
thoughts

64. 73
Level of automation
Controller
Business Layer
Data Layer
Model
View
Browser
automation
Trigger behaviour
through controller
Assert behaviour
on model, db, ..
Setup pre-conditions
through service
interfaces
Out-of-process
In-process

65. 74
Performance
• Grouping tests
• Current WIP
• Completed work
• Database
• In-memory
• Templates for setup
• Parallel execution
• Smart execution order

66. 75
Internal vs. external DSL
Example Source:
Liz Keogh
https://github.com/lunivore/tictactoe-java/blob/master/
scenarios/com/lunivore/tictactoe/scenarios/
Three_in_a_row_wins.java

67. 76
Non-functional acceptance criteria
Given there are 100,000 users registered on the system
When I create a new account
Then I should be taken to my dashboard within 5ms
Given 1000 users are hitting the homepage
simultaneously
Then each user should get a response within 2ms
Matt Wynne
http://blog.mattwynne.net/2012/03/13/using-cucumber-for-load-testing

68. 77
Tools

69. 78
Gherkin automation tools
www.cukes.info
www.behat.org
Ruby, Java, JavaScript, C++
www.specflow.org
.NET, Mono, Silverlight, WP7
PHP

70. 79

71. 80
Books
Gojko Adzic
Bridiging the
Communication Gap
Gojko Adzic
Specification by
Example
Elisabeth Hendrickson
Explore It!

72. 81
Christian Hassa: ch@techtalk.at - @chr99ha