“Specification by Example” is a set of process patterns that helps to validate the application for faster feedback and minimal documentation. With Specification by Example, teams write just enough documenta- tion to facilitate change effectively in short iterations or in flow-based development.

1. December 2011
Specifications For Enterprise Testing
Presenters
Sathyan Sethumadhavan & Shalabh Varma

2. Agenda
Where do we spend more time
A new approach for collaboration
Implement Specifications
Gauge Progress
Q&A
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

3. Do you experience any of this
Defects Leakage -
Missing Requirements
Frequent changes - Test and Requirements
Breaks existing documents becomes
functionalities stale
Long Regression Spends more time on
Cycles defect management and
triage
Less confidence -
Existing test
coverage
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

4. Do you spend more time here
Requirements
document
Creating test Fix production
Cases/Steps Defects
Traceability
Matrix
Defect
Management
Creating test plan
and strategy
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

5. Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

6. Specifications Driven Development
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

7. Can
EXAMPLES Become TESTS
Elaborate Verify
REQUIREMENTS
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

8. Specifications – Focus Area
Creating test
Requirements
plan and
strategy
Specifications
document
for both humans Specifying
Traceability and machines Collaborately
Fix Matrix
production
defects
Managing
Creating stale artifcats
test cases Tests as Validating
Documentation Frequently
Defect
management
and analysis
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

9. A Good Specification
Examples should be complete
Precise and Testable
True Specification – Not a script
About business functionality – Not software design
Self-explanatory and focused
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

10. A Good Specification
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

11. A Good Specification
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

12. Benefits
Distributed teams - Necessary artifacts and re-use artifacts
One-time specification – Referred by stakeholders and team
Artifacts remain as living documents
Focus on collaboration and communication gap
More time on exploratory/business testing
Time-to-market availability
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

13. Specifications – Key Process Patterns
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

14. Key Process Patterns
Derive Scope
Out Of Goals
Living Specifying
Documentation Collaborately
Validating Illustration using
Frequently examples
Automate without Refining
changing specification Specification
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

15. Key Process Patterns - Goals
Understand customer’s
business requirements
Derive Scope
Out Of Goals
Living Specifying Understand the “Why” and
Documentation Collaborately
“Who”
Validating
Frequently
Illustration using
examples
Understand the intent and
value
Automate without Refining
changing specification Specification
Understand the Output
Team delivers complete feature
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

16. Key Process Patterns - Collaboration
Derive Scope Out
Of Goals All Team Three
Living Specifying
Workshop Amigos
Documentation Collaborately
Validating Illustration using
Frequently examples
Ad-Hoc Write +
Conversation Review
Automate without Refining
changing specification Specification
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

17. Key Process Patterns - Examples
Requirements - Don’t provide full and
unambiguous context
Derive Scope Out
Of Goals
Living Specifying Don’t have Yes/No answers in your
Documentation Collaborately
example
Validating
Frequently
Illustration
using examples
Ask for alternate way for validation
Automate without Refining Avoid making up your own data
changing specification Specification
Avoid temptation to explore all
possibility
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

18. Key Process Patterns - Refining
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

19. Key Process Patterns - Refining
Don’t create workflow like descriptions
Derive Scope Out
Of Goals Don’t get trapped in user interface
Living Specifying details
Documentation Collaborately
Show and lets the user understand
Validating Illustration using
Frequently examples
Refined Examples = Acceptance
Automate without Refining criteria for delivery
changing specification Specification
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

20. Key Process Patterns - Refining
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

21. Key Process Patterns - Automation
Plan for automation upfront
Derive Scope Out
Of Goals Don’t delegate automation towards the
Living Specifying end
Documentation Collaborately
Avoid automating existing manual
Validating
Frequently
Illustration using
examples
scripts
Automate without
changing
Refining Automate along system boundaries with
Specification
specification
necessary context
Automated Specification = Executable
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.
Specification

22. Key Process Patterns - Automation
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

23. Key Process Patterns - Validation
Fix the most annoying thing, fix it and
repeat
Derive Scope Out
Of Goals
Living Specifying Simpler test doubles for external
Documentation Collaborately
systems
Validating
Frequently
Illustration using
examples
Isolated environment and fully
automated deployments
Automate without Refining
changing specification Specification
Look for ways for faster feedback
Don’t just disable failing tests
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

24. Key Process Patterns – Living
Documentation Don’t create long specifications
Derive Scope Out
Of Goals Avoid using technical concepts in tests
Living Specifying
Documentation Collaborately
Base specification on personas
Validating Illustration using
Frequently examples Evolve the ubiquitous language and
use it consistently.
Automate without Refining
changing specification Specification
Organize based on functional areas
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

25. Implementing Specifications
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

26. Greenfield
Implement as part of the big
process change
Use TDD as a first step towards
Specifications
Follow the key process patterns
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

27. Existing Projects
Start with automated tests
Don’t talk process or jargon words
Don’t make test automation end goal
Our Focus - Collaboration and process-
change
Executable specification in the same
codebase
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

28. Gauge Progress
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

29. Gauge Progress
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

30. Take Away
Distributed teams - Necessary artifacts and re-use artifacts
One-time specification – Referred by stakeholders and team
Artifacts remain as living documents
Focus on collaboration and communication gap
More time on exploratory/business testing
Time-to-market availability
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

31. Case Studies
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

32. Case Studies
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

33. Tools
Twist - http://studios.thoughtworks.com/twist-agile-test-
automation/
Cucumber – http://cukes.info
Concordion – http://www.concordion.org
FitNesse - http://fitnesse.org
Jbehave - http://jbehave.org
GreenPepper: http://www.greenpeppersoftware.com.
Robot Framework: http://www.robotframework.org
SpecFlow: http://www.specflow.org
TextTest: http://www.texttest.org
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

34. Contact us
Sathyan Sethumadhavan
satnair@thoughtworks.com
Shalabh Varma
svarma@thoughtworks.com

35. References
① Specification by example and Manning the specification-
Gojko Adzic
② Projects implemented SDD in real-time
Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.