We have experience with testing projects, both large and small. Sometimes our test estimates are accurate—and sometimes they’re not. We often miss deadlines because there are no defined criteria used to create our estimates. Sometimes we miss our schedules due to crunched testing timelines. Shyam Sunder briefly describes the different test estimation techniques including Simple, Medium, Complex; Top Down, Bottom Up; and Test Point Analysis. To assist in better estimating in the future, Shyam has prepared test estimation templates and guidelines, which can significantly help organizations in proper estimation of testing projects. Through his work, effort and schedule variations have significantly improved from ±60 percent to ±2 percent. Shyam explains the test estimation templates in detail and demonstrates how to choose the estimation templates for your organization’s software development process. Learn why effective software test estimation techniques help in tracking and controlling cost/effort overruns significantly.

1.
W7
Test
Management
5/10/17
13:45
Effective
Test
Estimation
Presented
by:
Shyam
Sunder
Sidra
Medical
and
Research
Center
Brought
to
you
by:
350
Corporate
Way,
Suite
400,
Orange
Park,
FL
32073
888-­‐-­‐-­‐268-­‐-­‐-­‐8770
·∙·∙
904-­‐-­‐-­‐278-­‐-­‐-­‐0524
-­‐
info@techwell.com
-­‐
http://www.starwest.techwell.com/

2.
Shyam
Sunder
A
PMP®
Certified
Project
Manager
with
19
years
of
IT
experience,
Shyam
Sunder
is
currently
working
at
Sidra
Medical
&
Research
Center
(Qatar)
and
has
worked
in
various
reputed
organizations
like
Dell,
United
Health
Group,
HCL
etc.
previously.
Shyam's
strengths
are
in
the
areas
of
Project
Management,
Software
Quality,
and
Client
relationship
management.
He
has
been
actively
involved
in
IV&V
along
with
his
IT
delivery
projects.
With
his
consistent
track
record
of
successful
product
introduction
and
implementation,
Shyam
is
productive
both
as
an
individual
contributor
and
project
manager.
He
regularly
contributes
papers,
presentations,
and
workshops
to
IT
forums.
Contact
Shyam
on
LinkedIn
and
at
shyamconf@gmail.com.

3. Test Estimation
SHYAM SUNDER - PMP, CSM, 6σGB
Sidra Medical & Research Center
shyamconf@gmail.com

4. Agenda
v Software Test Estimation
v How do we do Estimations ?
v Types of Test Estimations
v Software Test Estimation - Overview
v Issues while Estimating
v Software Test Estimation - Techniques
v Conclusion

5. Software Test Estimation
v Estimation is a prediction based on probabilistic assignments
v Software estimation is a continuous process, which should be
followed and used through out the project life cycle
v Effective software estimation helps track and control cost/effort
overruns
v Estimations covers following broad areas
Ø Estimate size
Ø Estimate cost & effort
Ø Determine the schedule
Ø Assess risks

6. How Can We do Test Estimations?
Test
Estimates
Test
Estimation Model
Test team
Product Exp.
Test Capability
…
Quality
Test Specifications
Test Size
/ Complexity
…
Test Platform
Test
Environment
Test Tools
…

7. Types of Test Estimation
Manual Test Execution Effort Estimation
The specifications
of a given test suite
23.5
man-hours
Estimated test
execution effort
Tools, Experience, ...
Test Automation Effort Estimation
The specifications
of a given test suite
APIs, Experience, ...
23.5
man-hours
Estimated test
automation effort
Test Coverage x Execution Effort Analysis
The specifications
of a given test suite
Maximum Effort
(optional)
Quality Criteria
Tests sorted to maximize
coverage with minimal effort
…
Tests to not
execute
Cost-benefit Analysis to Prioritize Tests for Automation
The specifications
of a given test suite
APIs, Experience, ...
Tools, Experience, ...
Tests to execute
manually
Frequency of
execution
Tests to automate
…

8. Software Test Estimation - Overview
Fine tune the
estimation process
Re-Estimate
Estimate Test Size
Estimate Cost & Effort
Determine Schedule
Track & Compare
Actual / Estimated
Budget
Baseline Data
Release Roadmap
Type of testing
Application size
INPUTS

9. Why Good Estimation is Needed and Why it Fails?
v Why good estimates are important?
Ø Testing is often blamed for late delivery
Ø Testing time is ”squeezed”
Ø It promotes early risk assessment
v Reasons for poor estimates?
Ø Incomplete/ambiguous requirements
Ø New technology
Ø Skills (Testers & developers)
Ø Project delays
Ø Poor environments

10. Issues While Estimating
v Major Issues
Ø Estimation method / Process
Ø Environment & Tools
Ø Testing to be performed
Ø Complexity of the application under tests
v Minor Issues
Ø Test Resource
Ø Other factors

11. Issues While Estimating - Process
v Requirement Stability
v Change Requests
v Finalize types of testing
v Follow the test process
v Non-availability of test cases and test data
v Timely reviews of the test cases and other artifacts
v Coordination with various teams/modules/interfaces
v Scope changes

12. Issues While Estimating – Environment & Tools
v Separate Environment
v Environment not similar to the deployment environment.
v Downtime of the environment available during testing
v Availability of test management tools
v Availability of test automation tools

13. Issues While Estimating – Environment & Tools
v Separate Environment
v Environment not similar to the deployment environment.
v Downtime of the environment available during testing
v Availability of test management tools
v Availability of test automation tools

14. Issues While Estimating – Testing Resources
v Management commitment towards completion and following the test life cycle
v Common and realistic expectation toward the testing goal from all the
stakeholder in project
v Availability of key resources
v Application knowledge among the test team
v Connect / Attitude between the development and testing team
v Correct resolution on the defect fixes
v Clear Communication

15. Issues While Estimating – Others factors
v Complexity of the application under tests
v Incorrect assumptions during estimation
v Ownership of testing, Independent Development Vendor Customer
v Timeline assigned for testing
v Development timeline and release of code for testing
v Availability of correct test data during test execution
v Geographical location of testing team involved
v Not doing periodic re-estimation

16. Software Test Estimation - Techniques
v Simple Medium Complex (SMC) Method
v Top Down
v Bottom Up
v Test Point Analysis (TPA)

17. Software Test Estimation – SMC Method
This model will consider the test functions / test conditions and their
complexities (Simple, Medium, Complex) as the basis for estimation
and the effort involved for the following test activities can be
estimated using this model.
Following test activities could be covered
• Test Initiation
• Test Planning & Design
• Test Execution
• Test Closure activities

18. SMC Method( Contd.. )
Test Initiation :
Effort estimate for the following activities can be done using SMC model under Initiation Phase :
v Knowledge Transfer , Application Familiarity, Requirements Analysis.
Test Planning and Design:
Effort estimate for the following activities can be done using SMC model under Test Planning & Design Phase:
v Test Plan ; Preparation of Scenarios, Test Cases, Test Data.
v Preparation of Execution Plan
v Test Ware Re-work & Reviews & Prepare and Review of Zero-day checklist
Test Execution :
Effort estimate for the following activities can be done using SMC model under Test Execution Phase :
v Creation of test bed & Test Execution ;Review of Defects & Update the Defect/Execution report
Test Closure :
Effort estimate for the following activities can be done using SMC model under test closure Phase:
v Closure Metrics Preparation ,Closure meeting
v Archive project data (Project Closure Activities)

19. Software Test Estimation – Top Down Method
v The Overall effort estimate for the project is determined first in FP
or Line of code method.
v The effort for various activities are determined.
v The method is more applicable to
Ø Early cost estimate (High level estimate)
Ø When only global properties are known
v The estimation procedure is as follows:
Ø Get the total size in FP
Ø Define the lower level project test component.
Ø Based on experience and productivity data from previous projects,
obtain the effort estimate
§ Overall effort estimate = productivity *size

20. Software Test Estimation – Bottom Up Method
v It is also known as “divide and conquer” technique
v It is hierarchical decomposition of the test effort into stages, activities
and tasks.
Ø Planning
Ø Test environment & configuration
Ø Test case creation
Ø Test execution
v Again decompose the above activity in smaller packages which can be
estimated in short period of time.
v Estimate the total effort by understanding the duration and effort of
each activity.

21. Dynamic Test Points Static Test Points
Environment Factor
Total Test Points
Productivity Factor
Primary Test Hours
Total Test Hours
Control Factor
1 2
4
3
5
Software Test Estimation –Test Point Analysis

22. Test Point Analysis(TPA) – Contd..
Test Point Analysis can be used to objectively prepare an estimate for black box
testing (excluding performance testing). Test Case Point Analysis methodology is
based on Test Case Points. Test Case Point is a Verification Point used to verify
that the value on AUT matches with the expected value. This O/p value can be I/p
data for other verification points.
v Test Points analysis is a method of estimation, which helps independent
testing projects estimate effort required for different phases of testing.
v TPA is concerned only with the measurable quality characteristics which fall
within the scope of acceptance testing and/or system testing.
v TPA is in principle analyst-independent. In other words, if two different
people performed a TPA of the same information system, they should obtain
the same result.
v TPA depends on the availability of a function point count
v For TPA purposes, the test team's matter knowledge is not treated as a
variable which affects the amount of work involved in the tests
v More Suitable for projects which have more planned releases and not adhoc

23. Test Point Analysis(TPA) – Contd..
v Experience with the TPA technique suggests that the following percentages are generally
appropriate:
Test Phase %
Test Scripting Preparation
(includes Functional Understanding)
10
Specification
(includes Test Conditions, Test Data identification, Test Script preparation)
40
Test Execution Test Script Execution and Defect Management
(includes Smoke, System, Integration, End to End and Regression Test)
45
Completion
(includes UAT)
5

24. The estimation technique guidelines explained in the earlier sections
can be enhanced to cover the various environmental factors. Pilot
these guidelines for couple of projects in your organization and
compare the estimated effort and actual effort. As we get
proficient with its implementation we will find that Estimated
and Actual efforts are getting closer which will result in better
execution of the testing projects.
Conclusion

25. Please feel free to contact me at
eMail : shyamconf@gmail.com
Twitter : shyam_sunder_30
Linkedin : sundershyam30