This is not a deck about how or why to practice TDD. Based upon research conducted, I outline the most common objections to TDD and describe how to refute (or more properly rebut), avoid or mitigate each of them. The coverage acknowledges that there are risks inherent to all techniques and does not promote the idea that TDD is some kind of silver bullet.
2. IBM Software Group | Rational software
Agenda
Motivation
Education
Inappropriate project
Cultural
Time pressures
Unclear benefits
2
Innovation for a smarter planet
3. IBM Software Group | Rational software
Refutations. Really?
re·fute
/rɪˈfyut Show Spelled
/
verb (used with object), re·fut·ed, re·fut·ing.
1. to prove to be false or erroneous, as an opinion or
charge.
2. to prove (a person) to be in error.
3
Innovation for a smarter planet
4. IBM Software Group | Rational software
TDD – in one slide
Test First predates XP and Agile
TDD has been around for a while
One of XP‟s core practices
Widely referenced
Still quite controversial
I am not going to describe TDD in any detail
Red/Green/Refactor (and variants)
„T‟ for Test has unfortunate connotations
First „D‟ definitely stands for Driven
Second „D‟ may stand for Development or Design
4
Innovation for a smarter planet
6. IBM Software Group | Rational software
Survey
Simple survey ran for 3 months
Jan – Mar 2012
260 unique respondents
896 objections
“What reasons are there not to practice TDD”
“The question wreaks of elitism - it should instead be „what reasons are
there to practice TDD‟”.
Free text made for a wide range of responses
Harder to analyse
Split into 18 types, grouped into 5 categories
6
Innovation for a smarter planet
7. IBM Software Group | Rational software
Results
Education
Time
Benefits
Project
Cultural
7
Innovation for a smarter planet
8. IBM Software Group | Rational software
Breakdown
Education (223)
Lack of Practice, Lack of Investment
Project (194)
Domain, Legacy, Environment, Tooling
Cultural (179)
Management, Team, Fanaticism, Demarcation, Egotism
Time (174)
Slowness of development, maintenance and execution
Benefits (120)
Lack of Proof & Experience, Alternatives
8
Innovation for a smarter planet
9. IBM Software Group | Rational software
Interconnectedness of all things
Classification is subjective
Many responses could be categorised several ways
Respondents had different points of view
Their own objections
Objections they had heard others express
Further analysis required
Follow-up survey(s)
9
Innovation for a smarter planet
10. IBM Software Group | Rational software
10
Innovation for a smarter planet
11. IBM Software Group | Rational software
Software Crisis!
The required techniques of effective reasoning
are pretty formal, but as long as programming is
done by people that don't master them, the
software crisis will remain with us and will be
considered an incurable disease. And you know
what incurable diseases do: they invite the
quacks and charlatans in, who in this case take
the form of Software Engineering gurus.
- Dijkstra (ACM Turing Award Lecture 1972)
11
Innovation for a smarter planet
12. IBM Software Group | Rational software
Fanaticism and Fluff
The steps of FDD are simple:
1. Take a tiny piece of fluff from the plate and put it on your
head, holding your head quite still to ensure that the fluff
does not fall off your hair.
2. Write a line of code.
3. Say, “I am the Fluff Lord, within the Dominion Of The
Fluffists.”
4. Repeat.
That’s it.
Seriously, that’s all there is to it.
12
Innovation for a smarter planet
13. IBM Software Group | Rational software
Agenda
Motivation
Education
Inappropriate project
Cultural
Time pressures
Unclear benefits
Wrap up
13
Innovation for a smarter planet
14. IBM Software Group | Rational software
Hard to learn
“TDD is hard.”
Yes, it is
Any technique is hard to master
Fundamental changes are hard to adopt
10,000 hours of practice
Did you stop trying to ride a bicycle when you fell off?
14
Innovation for a smarter planet
15. IBM Software Group | Rational software
Where to start
“Sheer lack of knowledge on how to approach it appropriately.
There are too many bowling examples and not enough
practicality.”
How do you usually learn something new?
Unit Testing skills are foundational
Coding Dojo to practice TDD
http://cyber-dojo.com
Better used as a group, but still useful solo
15
Innovation for a smarter planet
16. IBM Software Group | Rational software
Tests before code
“The compiler complains if I write tests before the code”
First step to get to green is to make the test compile
Only then can you move on to make the test pass
16
Innovation for a smarter planet
17. IBM Software Group | Rational software
Tests will be buggy
“Your test code is just your production code, written from the
other end (i.e. just as complex and likely to have bugs)”
No process is infallible
“To err is human …”
Safety in numbers
Pair programming
Peer review
17
Innovation for a smarter planet
18. IBM Software Group | Rational software
Unfamiliar architectural style
“How will we ever know what‟s going on if we can't see all of
the code at once?”
TDD tends to lead to small, concise implementations
Low coupling
High cohesion
“Proliferation” of interfaces
Literate-ish programming
18
Innovation for a smarter planet
19. IBM Software Group | Rational software
No design improvement
“My observations of code from TDD-based projects show no
significant improvement in architecture, security, code
style, testability, etc. over other projects built with testing in
mind.”
TDD ensures testability
Tendency for smaller decoupled composition
Other design concerns (e.g. security, performance etc.) not
addressed by TDD
19
Innovation for a smarter planet
20. IBM Software Group | Rational software
Learning several things at once
“In a new technology, it's too difficult to learn how to TDD as
well as how to master that technology.“
Even if you are experienced with TDD, it changes your
perspective on the new technology
Idioms
Tooling
Kent Beck suggests reimplementing xUnit in the new
language
20
Innovation for a smarter planet
21. IBM Software Group | Rational software
Discipline
“TDD is a discipline and a work habit. It's very difficult to
establish the habit.”
Habits are hard to form, but also hard to break.
Nothing works better than positive feedback
There are many self-help guides available
Switch – Chip & Dan Heath
Drive – Dan Pink
21
Innovation for a smarter planet
22. IBM Software Group | Rational software
Agenda
Motivation
Education
Inappropriate project
Cultural
Time pressures
Unclear benefits
Wrap up
22
Innovation for a smarter planet
23. IBM Software Group | Rational software
Too simple
"This code is too simple to need a test“
Then the tests will be simple too
What‟s simple to you may be opaque to others
As the code evolves, tests can help keep it simple
23
Innovation for a smarter planet
24. IBM Software Group | Rational software
Just a spike
“Because you're prototyping an idea and it's much faster to
spike without tests. If you do end up using the code then you
can write tests and refactor.”
Use the right tool – TDD is not mandatory
Discipline to ensure spike does not mutate
“Write one to throw away”
TDD the spike anyway!
24
Innovation for a smarter planet
25. IBM Software Group | Rational software
Not useful for all aspects of design
“I don't think that deriving a design by writing tests is a useful
practice. Tests by themselves cannot cover many aspects of
a design (designing for concurrency and performance in
particular by writing tests is something that I've never seen
anyone do).”
TDD is not the only tool in the toolbox
Probably wrong tool for performance testing
Can be applied to concurrency testing (but not recommended)
25
Innovation for a smarter planet
26. IBM Software Group | Rational software
Not useful for functional/declarative programming
“Because it is largely adequate in imperative programming
and not when you go functional and declarative - when code
reads like specification.”
Orthogonal.
Absence of side effects makes testing MORE effective.
Discuss!
26
Innovation for a smarter planet
27. IBM Software Group | Rational software
Legacy code
“Difficulty starting with legacy code. A simple change done
through TDD can take orders of magnitude longer due to a
need to redesign toward testability. ”
Working Effectively With Legacy Code – Feathers
Small, conservative steps, eventually tame fear and doubt
Consider less intrusive approaches initially
e.g. TextTest – texttest.org
27
Innovation for a smarter planet
28. IBM Software Group | Rational software
Insufficient tooling
“There is no unit-test framework for the language I'm
using, and I don't have time/inclination to develop one
myself.”
There probably is.
You might not need one
“TDD in C” – Olve Maudal
28
Innovation for a smarter planet
29. IBM Software Group | Rational software
GUI
“I do a lot of UI code and don't really know how to properly do
that with TDD.”
“Subcutaneous” testing
Presenter First pattern
Based on MVC/MVP patterns
Very shallow view, with no business logic
Stateless presenter orchestrates interaction between view and model
29
Innovation for a smarter planet
30. IBM Software Group | Rational software
Excessive coupling to data
“A test expresses a unit of change as data that fits the needed
computation. Creating that data is harder than writing the
program that accomplishes the change.”
Unit tests preferably utilise „small‟ amounts of data
For domains where this is not possible
You will need large amounts of data irrespective of test approach
Bootstrap process with constructed data
Continue with captured data
30
Innovation for a smarter planet
31. IBM Software Group | Rational software
Agenda
Motivation
Education
Inappropriate project
Cultural
Time pressures
Unclear benefits
Wrap up
31
Innovation for a smarter planet
32. IBM Software Group | Rational software
Management antipathy
“Management want results NOW; very much willing to clean
up small oversights later. 'Close is good enough' attitude.”
Management are generally result focused
Previous bad experiences affect appetite for change
Both ways!
Start small
Need to demonstrate benefit to „bottom line‟
32
Innovation for a smarter planet
33. IBM Software Group | Rational software
Team resistance
“Teammates are not prepared. They do not have enough
knowledge or experience in unit testing.”
“No one tells me how to program!”
Changing behaviours is hard
“Fearless Change” has many patterns for introducing change
33
Innovation for a smarter planet
34. IBM Software Group | Rational software
I never make mistakes
“I already know what the code needs to do and it's low risk. I
don't need a test for it.”
“My first design idea is always perfectly good.”
“My code always works the first time.”
Do people really believe this?
Even if they do, is everyone that ever touches the code going
to be so talented?
34
Innovation for a smarter planet
35. IBM Software Group | Rational software
Testing is for testers
“We don't need TDD, we've got a QA department. They'll find
the bugs for us.”
TDD is not just about testing
Drives design
Refactoring/regression safety net
Write tests to explore EVERY defect you find
Living documentation
Helps future developers understand intended behaviours
35
Innovation for a smarter planet
36. IBM Software Group | Rational software
All TDD-ers are fanatics
“It has become a cult. Its advocates have made it antithetical
to „Individuals and interactions over processes and tools‟. It is
evangelized through coercion and browbeating --
necessarily, as it isn't compelling on it's own.”
“Someone needs to tell unclebob that he might be right, but
he's part of the problem - to non-modern coders (which felt
like the majority last time I looked) he comes across like a
raving loony.”
36
Innovation for a smarter planet
37. IBM Software Group | Rational software
Fluff (continued)
If at any time you even THINK about writing a line of code
before putting fluff on your head, then you‟ve to delete all your
code, shake all the fluff from your head onto the plate and start
all over again.
I have been practicing FDD for 12 years now; sometimes in the
office, I have so much fluff on my head that my boss thinks I‟m
a hay-stack, only made of fluff. A sort-of fluff-stack, if you will.
But one thing is beyond doubt: the code I write, when I‟m in this
fluff-zone, is the most high-quality code that anyone has ever
seen.
37
Innovation for a smarter planet
38. IBM Software Group | Rational software
Agenda
Motivation
Education
Inappropriate project
Cultural
Time pressures
Unclear benefits
Wrap up
38
Innovation for a smarter planet
39. IBM Software Group | Rational software
More code to write
“Takes too much time to write test.”
How will your code get tested?
Test team?
Customers & Stack Trace Driven Development?
How much time will be spent figuring out how to make
changes later?
What proportion of your time do you actually spend coding?
39
Innovation for a smarter planet
40. IBM Software Group | Rational software
More code to maintain
“If I change something it will break a whole bunch of tests that
I will have to fix and it will be more work for me in the end
than just verifying my changes manually.”
It is USEFUL to know when behaviour changes
Foundational Unit Testing skills reduce brittleness
next 6 slides
40
Innovation for a smarter planet
41. IBM Software Group | Rational software
Testability
Testability needs to be designed in
TDD ensures code is testable
Code with hidden dependencies is hard to test
Dependency Injection/Inversion
Pass dependencies into code under test
Write factories that permit injection of test doubles
Interfaces should be cohesive
Wide interfaces encourage unnecessary coupling
Avoid globals, singletons etc.
Retro-fitting unit tests is hard
Take small steps
Introduce a „seam‟ – c.f. Working Effectively with Legacy Code
41
Innovation for a smarter planet
42. IBM Software Group | Rational software
Necessity
Test observable behaviour
Don‟t modify encapsulation to aid testing
If a behaviour isn‟t observable through the public interface what is it for?
Don‟t slavishly write one test per method
Test behaviours
Some methods may not need any dedicated tests
Methods that implement useful behaviours may need many tests
Choose test variants carefully
Edge conditions
Invalid inputs
Multiple invocations
Error signalling
42
Innovation for a smarter planet
43. IBM Software Group | Rational software
Granularity
Test a SINGLE observable behaviour
It is tempting to combine related behaviours in a single test – DON‟T
… even if EXACTLY the same steps are needed
public void shouldSortTwoStringsAndReportCorrectSize() {
SortedSet<String> animals = new TreeSet<String>();
animals.add(“Zebra”);
animals.add(“Anteater”);
assertEquals(2, animals.size());
assertEquals(“Anteater”, animals.first());
assertEquals(“Zebra”, animals.last());
}
43
Innovation for a smarter planet
44. IBM Software Group | Rational software
Understandability
Name tests to describe the behaviour under test
Describe nature of the test
Is it checking that preconditions are enforced?
Is a dependency going to signal an error?
Long names are fine – you only type them once
Be precise
shouldReturnCorrectValue is not a good name for a test
shouldReturnCorrectSumOfTwoIntegersWithoutOverflow
should_return_correct_sum_of_two_integers_without_overflow
When a test fails you want to know WHAT WENT WRONG
You don‟t want to reverse engineer the test
You don‟t want to run smaller tests to isolate the failure
44
Innovation for a smarter planet
45. IBM Software Group | Rational software
Maintainability
Unit Tests should be written to same quality as Production code
Tests will be maintained and read just as often as production code
Code is communication to other developers not just a compiler
Organise tests into cohesive suites
Refactor tests to avoid duplication
Use suites to perform common set up/tear down operations
Extract common code into methods
Extract common functionality into classes
Remove redundant tests
45
Innovation for a smarter planet
46. IBM Software Group | Rational software
Reiteration: 5 -ities
Testability
Necessity
Granularity
Understandability
Maintainability
MUNGT ?
TeNGUM?
46
Innovation for a smarter planet
47. IBM Software Group | Rational software
Mock-based tests rot
“Unit-test mock/stub assumptions rots”
Use collaboration and contract tests – J.B. Rainsberger
Collaboration tests make assumptions about the contract; contract tests
try to justify those assumptions
A stub in a collaboration test must correspond to an expected result in a
contract test
An expectation in a collaboration test must correspond to an action in a
contract test
47
Innovation for a smarter planet
48. IBM Software Group | Rational software
That library is third party
“API layers above and below you don't provide adequate
mocks. Testing against the „real thing‟ is hard/impossible.”
“Don't mock types you don't own” – Joe Walnes
Write adapters that provide a domain specific API
Write acceptance tests that verify the library‟s behaviour
Run whenever adopting a new version
48
Innovation for a smarter planet
49. IBM Software Group | Rational software
Slow execution
“Running the tests takes too long.”
For TDD to work tests need to (build and) execute in seconds
Some environments need careful configuration
C/C++
Rails
Are they really Unit Tests? (See next slide)
49
Innovation for a smarter planet
50. IBM Software Group | Rational software
A test is not a unit test if:
It talks to the database
It communicates across the network
It touches the file system
It can‟t run at the same time as other unit tests
You have to do special things to your environment (such as
editing config files) to run it
(Michael Feathers‟ blog, 2005)
50
Innovation for a smarter planet
51. IBM Software Group | Rational software
Agenda
Motivation
Education
Inappropriate project
Cultural
Time pressures
Unclear benefits
Wrap up
51
Innovation for a smarter planet
52. IBM Software Group | Rational software
Absence of research
“No scientific proof of it actually having any benefit (compared
to just code reviews, pair programming or etc.) for the same
amount of time”
“It's not practical for the kind of work I do. By which I mean, it
cannot be conclusively shown to provide tangible benefits that
outweigh the perceived costs.”
“In an industry where todays must have offering is tomorrows
dustbin liner, it's often prudent to wait before trying the latest
snake oil!”
52
Innovation for a smarter planet
53. IBM Software Group | Rational software
Absence of research (cont.)
“We found that test-first students on average wrote more tests
and, in turn, students who wrote more tests tended to be
more productive. We also observed that the minimum quality
increased linearly with the number of programmer
tests, independent of the development strategy employed.”
On the Effectiveness of Test-first Approach to Programming
Proceedings of the IEEE Transactions on Software Engineering 2005
Lots of other papers are referenced from:
http://bradapp.blogspot.co.uk/2009/08/studies-on-effectiveness-of-tdd.html
53
Innovation for a smarter planet
54. IBM Software Group | Rational software
Our process is good enough already
“My team doesn't practice it. It is difficult to change existing
practices if they seem to work reasonably well.”
“I've personally been a part of four 'large' software projects
where active pursuit of a test suite was in play. None of the
four projects ever launched into production. The people I
worked with were smart and highly ambitious, yet they (we)
failed to launch.
“I've personally been a part of dozens of software projects
where active pursuit of a test suite was NOT in play. Most
every one of those projects launched into production in a
timely manner.”
54
Innovation for a smarter planet
55. IBM Software Group | Rational software
Test after
“It's easier to write the code first and the test after.”
“Writing tests after code is just as good.”
“Writing tests before code doesn't make sense.”
Testability
Rework needed if code not testable
Discipline
Will you really write the tests later?
55
Innovation for a smarter planet
56. IBM Software Group | Rational software
BDD
“Why practice TDD when you can practice BDD?”
BDD extends TDD
Involves stakeholders, not just developers
Even harder to implement.
56
Innovation for a smarter planet
57. IBM Software Group | Rational software
ATDD, BDD, GOOS
Many shades of outside-in, test-first development
57
Innovation for a smarter planet
58. IBM Software Group | Rational software
None of these are enough either
Given the account is in credit
And the card is valid
And the dispenser contains cash
When the customer requests cash
Then check that the account is debited
And check that cash is dispensed
And check that the card is returned
“And check that nothing happens that shouldn’t happen and
everything else happens that should happen for all variations
of this scenario and all possible states of the ATM and all
possible states of the customer’s account and all possible
states of the rest of the database and all possible states of the
system as a whole, and anything happening in the cloud that
should not matter but might matter.”
58
Innovation for a smarter planet
59. IBM Software Group | Rational software
Manual testing
“It is much faster to just think about the proper
implementation, write the code and manually test it.”
Until you need to do it again
… and again
Manual scripts
Error prone
Time consuming
Expensive
59
Innovation for a smarter planet
60. IBM Software Group | Rational software
Agenda
Motivation
Education
Inappropriate project
Cultural
Time pressures
Unclear benefits
Wrap up
60
Innovation for a smarter planet
61. IBM Software Group | Rational software
Not sufficient, but necessary?
TDD is not enough. Consider:
Acceptance
Usability
Performance / Stress
Exploratory
Systems have been, and continue to be, successfully
delivered without TDD.
But they have also been delivered without designs, documentation, or
…. testing of any kind.
61
Innovation for a smarter planet
62. IBM Software Group | Rational software
Conclusions
There are many impediments to adopting TDD
Benefits are not universally accepted
TDD is not a “Silver Bullet”
62
Innovation for a smarter planet
63. IBM Software Group | Rational software
Essential Reading
Test Driven Development by Example – Kent Beck
Growing Object Oriented Software Guided By Tests – Steve
Freeman/Nat Pryce
Working With Legacy Code – Michael Feathers
Fearless Change – Mary Lynn Manns/Linda Rising
JUnit Recipes – J.B. Rainsberger
63
Innovation for a smarter planet
64. IBM Software Group | Rational software
Other references
Fluff Driven Development:
http://www.threeriversinstitute.org/blog/?p=594
Presenter First:
http://atomicobject.com/files/PresenterFirstAgile2006.pdf
TDD in C:
http://www.pvv.org/~oma/TDDinC_Smidig2007.pdf
On the effectiveness of Test-first approach to Programming:
http://nparc.cisti-icist.nrc-
cnrc.gc.ca/npsi/ctrl?action=rtdoc&an=5763742&article=0&lang=
en&ext=pdf
64
Innovation for a smarter planet
Notas do Editor
1987 – “Test, then code” – 4th Int. Conf. Software Testing, Washington, DC
Not enough personnelCancelled projectsOverrunsLarge projects more likely to overrun¾ all large systems have operational failures
Gladwell - outliers
Literate programming is an approach to programming introduced by Donald Knuth as an alternative to the structured programming paradigm of the 1970s
Emergent Design is a consistent topic in Agile Software Development, as a result of the methodology's focus on delivering small pieces of working code with business value. With Emergent Design, a development organization starts delivering functionality and lets the design emerge. Development will take a piece of functionality A and implement it using best practices and proper test coverage and then move on to delivering functionality B. Once B is built, or while it is being built, the organization will look at what A and B have in common and refactor out the commonality, allowing the design to emerge. This process continues as the organization continually delivers functionality. At the end of an agile or scrum release cycle, Development is left with the smallest set of the design needed, as opposed to the design that could have been anticipated in advance. The end result is a smaller code base, which naturally has less room for defects and a lower cost of maintenance[1].As Emergent Design is heavily dependent upon Refactoring, practicing Emergent Design without a comfortable set of unit tests is considered an irresponsible practice.
See answer by Cellfish -http://stackoverflow.com/questions/1715822/unit-test-for-thread-safe-ness/1746652