1. Introduction to Automated Testing
by Lars Thorup, ZeaLake
May 26th 2011

2. Who is Lars Thorup?
• Software developer
• Coach: Teaching TDD and
automated testing
• Advisor: Assesses software
projects and companies
• Founder and CEO of
BestBrains and ZeaLake

3. Why are we here today?
• How do we do automated testing?
– Write test programs
– Run the tests automatically
• Why should we write tests?
– Enjoy more efficient and predictable course of development
– Find and fix bugs fast
– Prevent bugs from reappearing
– Improve the design of our software

4. Different kinds of automated tests
• Unit tests
– Tests individual pieces of code and the interaction between code blocks
• System tests
– Tests the entire system against the requirements
• Performance tests
– Tests non functional requirements

5. Unit tests or system tests?
• Unit tests are efficient
– Fast to run (a few seconds)
– Robust and predictable
– Easy to write
– Is written together with the code it is testing
• System tests are thorough
– Tests all layers together
– Most efficient way to create a set of tests for existing code

6. Can we automate performance tests?
• Performance tests are brittle
– Tip: create performance trend curves instead

7. So, how do we actually do this?
• IsNumeric
– C#
– Ruby
• TransferFunds
– C++

8. How do we run the tests automatically?
• From our programming environment (IDE)
– Command line: make test
– Right click | Run Tests
• On every commit
– Setup a build server
• Hudson / Jenkins
• TeamCity
– Let the build server run all tests
– Get build notifications
– Keep the build green
• Fixing a broken build has priority over any other development task

9. How can tests help improve our design?
• The software design needs to evolve over time
• A refactoring modifies the design without changing behavior
• Tests ensure that behavior is not accidentally changed
• Without tests, refactoring is scary
– and with no refactoring, the design decays over time
• With tests, we have the courage to refactor
– so we continually keep our design healthy

10. What is good design?
• One element of good design is loose dependencies
– Use interfaces (for static languages)
– Inject dependencies
• Avoid this
• Do this instead

11. Are we wasting valuable developer time writing tests?
• No
• The time spent writing tests is not taken from the time spent
coding
– It is taken from the time otherwise spent on manual testing and debugging
• The cost of a bug keeps increasing until we fix it
• Find bugs fast
– Avoid losing customer confidence
– Free QA to do exploratory testing
so they find the hard-to-find bugs
– Spend less time trying to figure out
what is causing the bug and how to fix it
• Avoid spending time testing again

12. How do we get started?
• When we have a lot of existing code without tests
– Create a set of system tests to get a safety net
• When we are writing new code
– Write unit tests in conjunction with the new code
• Set up a standard test environment for our specific application
– Test data
• Automate the creation of standard testdata in a local database
– External dependencies
• Write stubs to use in the tests

13. What does a real-world project look like?
• wizerize.com
– Web application: C# and JavaScript
– 3 years of production
– 2-4 developers
• 40% test code, 60% production code (in lines of code)
• 71% code coverage of unit tests
• 614 unit tests – run in 1 minute
• 54 system tests – run in 20 minutes
• No functional errors seen by end users in production (yet)

14. Where can I read more?
• http://googletesting.blogspot.com/
• http://testdrivendeveloper.com/
• http://codesheriff.blogspot.com/

15. Which tools do we use?
Environment Tool
C# NUnit
JavaScript qUnit
HTML-based UI WatiN, Selenium
C++ CppUnit, googletest
Python unittest
Ruby Test::Unit
C check, cunit
Java JUnit
... ...