A gentle introduction to Acceptance Test Driven Development

1. Acceptance Test Driven
Development in practice
Steven Mak
steven@odd-e.com

2. What are we up to now?
Lost in translation
Do not explain why
Gaps discovered only until coding started
Cumulative effects of small misunderstandings
Inadequate and essentially flawed requirements and
specifications

3. Failing to meet actual needs
are obvious things really obvious?
Imperative requirements
Fulfilling specifications does not guarantee success

4. Meeting the needs with
Acceptance TDD
Acceptance Test Implementation
Requirement Feedback
Drive implementation of a requirement through a set of
automated, executable acceptance tests

5. ATDD in a Nutshell
Real-world examples to build a shared understanding of the
domain
Select a set of these examples to be a specification and an
acceptance test suite
Automate the verification of acceptance tests
Focus the software development effort on the acceptance
tests
Use the set of acceptance tests to facilitate discussion about
future change requests.

6. The ATDD cycle
•Implementation of tests usually
coding
occur before the code is done
testing
• Feature not done until test passes
Feature
Specification architecture
Done
Workshop customer documentation
other activities
• Activities happen in parallel
Developers
Testers
•Team clarifies and implement the feature
Example
Product Owner
Architect tests together
Technical writers
•Tests are executable at the end of the ATDD
Workshop

7. Benefits of ATDD
Comprehensible examples over complex formulas
Close Collaboration
Definition of Done
Trust and Commitment
Testing on system level

8. Specification by Examples
Use realistic examples to demonstrate differences in
possibilities instead of abstract requirements
Write specifications down as tables
Workflows:
Preconditions
Processing steps
Verifications

9. Examples, Tests, and Spec
can become
Examples Tests
ela
rify
bo
ve
ra
te
Requirements

10. Specification workshop
Ask the domain experts
Developers and testers should suggest examples of
edges or important issues for discussion
Ubiquitous language
Organise feedback to ensure shared understanding
Use facilitator to stay focused if needed

11. Distilling the specifications
Write tests collaboratively
Examples in a form close to what your automation tool can
understand
Keep tests in a form that is human-readable
Specification over Scripting, describe WHAT, not how
Acceptance tests to prevent defects, not to discover
Not necessarily automate anything
Acceptance tests only work when we can discuss them

12. Some considerations
User Interface
Easy?
Fragile?
Performance issues?
Boundary of Stub
Sufficiently close
Simulators?
Business logic
Not from developer perspective

13. Acceptance Test smells
Long tests
Parameters of calculation tests that always have the same
value
Similar test with minor differences
Tests that reflect the way code was written
Tests fail intermittently even though you didn’t change any
code
Interdependent tests, e.g. setup for others

14. Change
Use existing acceptance tests to discuss future
changes
Seek advices from customer to determine if it specifies
obsolete functionality when test fails
Automate periodic execution of regression tests with CI
Keep tests in the same version control as code

15. Tools
Table-based frameworks
FIT, http://fit.c2.com
RobotFramework, http://robotframework.org
Text-based frameworks
Exactor, http://exactor.sourceforge.net
TextTest, http://texttest.carmen.se

16. FIT
FIT stands for “Framework for Integrated Tests”
Most popular framework in-use
Table-based
Supporting languages like Java, C/C++, C#, Python, or
Ruby
FitLibrary to extend FIT

17. FIT in practice
Test doc
with tables
Customer writes a
Technical staff enhance
test document
the tables in the doc
containing examples
Test doc with
sanitised
tables
Technical staff
Executable Test implements fixture
classes
Test doc and
backing code
(e.g. Java)

18. FIT Document
Customers write acceptance tests in their own
language using their own tools.

19. Fixtures
Take information from the tables and turn them into
method calls on the actual application.
ColumnFixture - checking rules and calculations
ActionFixture - Step-by-step processing
RowFixture - Checking sets of data

20. Fixture example

21. Robot Framework
Python-based Keyword-driven test automation framework
Test libraries implemented either in Python or Java (with
remote library available in many other languages thru XML-
RPC)
Test cases are written in tabular format, save in HTML or TSV
files
Syntax similar to natural language
Users can create new keywords from existing ones and
contribute to the project

22. Preparing Test cases
Workflow tests - takes in initial state, action, verified
that the system behaved as expected
Test procedure using keywords
Keyword arguments
Test case name
Test Case Action Argument
Valid Login Open Login Page
Input Name demo
Input Password mode
Submit Credentials

23. Higher-level test cases
Free text suitable for communication even with non-
technical customers
Support given-when-then format in BDD

24. Data-driven test cases
Define a keyword which will take the input data and
prepare a table with test cases

25. Test case organisation
Simple way: Single HTML file containing all test cases
Test case tagging

26. Execution
Gathering test cases, reading and setting variables
Executing all actions for every test case
Providing global statistics
Writing the output in XML format
Generating report and log in HTML format

27. Sample execution result

28. Sample Test Report
in HTML format, showing all actions executed up to the
failing action, with fail message

29. Tested application Interface
Command line
OperatingSystem
SSHLibrary
Telnet library
Web
Robot Selenium
GUI
Swing GUI library

30. Adoption
Sense of achievement
Integrity
Openness
Right timing

31. Facilitate Adoption
Evangelise - increase awareness
Lower the bar - quick path to the first success
Train and educate - from general training to specific needs
Share and infect - sit together and sharing sessions
Coach and facilitate - learn about yourselves and cope with the findings
Involve others - know the stake-holder groups and give them roles

32. Ideal candidate to work with
Shared interest in success
Authority to make decision
Ability to understand implications
Ability to explain the domain

33. Variations - an escape?
Behaviour-driven development
Example-driven development
Executable specifications
Names do not matter, but underlying practices matter
Worthwhile to try if your business people do not like
“testing”

34. Organisational Challenges
Business Analysts
Great choice for the facilitator of the spec workshop
Working with Acceptance tests makes BA’s work easier
Testers
Help people avoid problems, not to discover them
Automation of acceptance test gives testers more time to test manual things
Developers
Spec workshop gives a good chance to discuss with domain experts
Both unit tests and acceptance tests are important

35. References
Bridging the Communication Gap
Gojko Adzic
Practical TDD and ATDD for Java Developers
Lasse Koskela
Agile Testing
Lisa Crispin and Janet Gregory

36. Thank you!
Steven Mak
Email: steven@odd-e.com
Twitter: http://twitter.com/stevenmak