2. Agenda
• Classes of Test Tools
▫ Test Management and Control Tools
▫ Test Specification Tools
▫ Tools for Static Testing
▫ Tools for Dynamic Test
▫ Tools for Non-functional Test
• Selecting Test Tools
• Introducing Test Tools
3. Test Management and Control Tools
• Administering test cases and results of their
execution
• Testing resource & schedule planning
• Advanced test management tools support
requirements-based testing; requirements
management; incident management; configuration
management
• Tool integration (req / test mgmt <-> test execution
<-> config mgmt)
• Generating test reports and test documentation.
4. Test Specification Tools
• Test (data) generators
▫ Database-based
▫ Code-based
▫ Interface-based
▫ Specification-based – the one generating expected
outputs!
5. Tools for Static Testing
• Review support
• Static analysis
• Model checking tools
6. Tools for Dynamic Test
• Debuggers (not really a test tool)
• Test drivers (test harnesses), test harness and stub generators
• Simulators
• Test robots (data-driven testing, command- or keyword-driven
testing, interaction method – object-oriented management of
test modules, so called interactions, in a DB)
• Comparators (compare something static with expected
results)
• Dynamic analysis tools
• Coverage analyzers (insert measurement code into the test
object prior to test run – make this code ‘covered’).
7. Tools for Non-functional Tests
• Load and performance test tools
• Monitors
• Security may be checked using even virus scanners
and firewalls (their logs).
8. Basics of Tools Introduction #1
• Automating chaos just gives faster chaos!
• Recommended order of tool introduction (by type):
▫ Incident management
▫ Configuration management
▫ Test planning
▫ Test execution
▫ Test specification
9. Basics of Tools Introduction #2
• Some time is necessary to learn the tool and
establish its use so productivity may even decline. It
is risky to introduce a new tool during ‘hot’ project
phrases.
• Without good test procedures or well-established
test methods, tools do NOT lead to the desired cost
reduction.
10. Test Tool Costs and Benefits
• Make a cost-benefit analysis of a tool taking into account
selection, acquisition and maintenance costs, as well as
time frame needed for the tool to start paying back.
• Benefits include not only time savings but also larger
number of found bugs and higher trustworthiness of
tests (so there are savings on development, support and
maintenance).
• Extra test programming effort does not pay back after
one test run but only after many regression test runs (if
tests are programmed for easy use and maintenance).
• Some tests (like performance test) can only be run
automatically.
11. Use of Tools
• Creative test activities can be supported by tools.
• Mechanical test execution can be automated.
12. Test Tool Selection Process
1. Requirement specification for the tool application.
2. Market research (creating an overview of possible
candidates).
3. Tool demonstrations and creating a short list.
4. Evaluating the tools on the short list.
5. Review results and selection of the tool.
13. Test Tool Introduction Success Factors
• Stepwise introduction (including a pilot project).
• Integration with processes.
• User training and continuous coaching.
• Make use experiences available to all users
(hints, tricks, FAQs, etc.).
• Monitoring of tool acceptance, gathering and
evaluating cost-benefit data.
14. Test Tool Introduction Steps
1. Execute a pilot project.
2. Evaluate the pilot project experiences.
3. Adapt the processes and implement rules for
usage.
4. Train the users.
5. Introduce the tool stepwise.
6. Offer accompanying coaching.
