This document discusses establishing a prediction model to predict testing effort and schedule for a software development organization. It suggests identifying controllable factors that impact testing from historical project data, such as defects injected at different stages. Prediction models would be established using these factors to predict testing effort and schedule. The models would help manage iteration schedules and system testing to meet goals like reducing defects and detecting more during testing. Ongoing data collection and refinement of models is important.

1. Whatprediction model should be？ Institute of Software Chinese Academy of Sciences Qing Wang

2. Why we use prediction model？ Understand the past behavious of process Control the present behavious of process Predict the future behavious of process 2

3. How to understand the past behavious？ Based on the historical data which has been collected from relative processes Apply appropriate measure and analysis techniques to analyze the data quantitatively. Try to find some nature characteristics Is stable？Nature bounds？ Does some quantitative relationship among process attributes？ Can be stratified /segmented or composed further？ 3

4. How to control the present behavious？ Based on the nature bounds (baselines) to establish the desired objectives of project Compare the present data to say if it is compliant with the nature bounds. Take proper action to correct the deviation Monitor and evaluate the effect of correct action 4

5. How to predict the future behavious？ Based on the quantitative relationship which established depend on the understanding of past behavious. Predict the interim outcome and final outcome of project to know if the project is going well toward the project objectives. 5

6. What is the essential ？ Differ from estimation model Deterministic, uncertainty, …… Use past and present data to predict future Some dependent X factors, predict independent Y outcomes The measurement for Xs and Y make sense Data collection timely and frequently Dynamical Statistical, probability, simulation-based 6

7. Must be careful when establish prediction model？ Use real world and relevant data Align the necessary project objectives SMART principle Specific Measurable Attainable Relevant Time-bound Cost-benefit balance Feedback and refinement 7

8. Practical Case – A CMMI ML4 organization Issue: the organization found there are more than 90% projects delay is due to test and fix bugs need much more time than estimated Question: How to predict the effort and schedule of test process? Method: Identify the factors which impact the effort and schedule of test process Establish a prediction model use these factors Measure and control these factors Predict the effort and schedule of test based on the factors when they were got

9. The process Each user case has been implemented by requirement analyze, design and coding reviews for each user case for each development activity were conducted respectively and defect data was recorded Each iteration has a unique testing and data was recorded The subsystem of each iteration was delivered to customer as a intermediate result, they must be on schedule and have good quality System testing was conducted when all iterations were completed (each release)

10. Process improvement objectives Goal G1: manage the schedule of each iteration and reduce the defect as soon as possible G2: manage the system test to detect and fix defects as many as possible

11. G1: in each iteration All defect data related requirement analysis, design and coding were collected before testing Can we find some quantitative relationship between these data and the effort of testing process? There are two primary activities in testing process Detect defects Fix defects

12. Get data from 16 history projects

13. Defect injection analysis - 1 13 Defect injected in requirement analysis Defect injected in design

14. Defect injection analysis - 2 Defect injected in coding The factor can be control , it can be used to establish prediction model Y is the effort of fix defects. If the predicted effort beyond estimate, they should assign more engineers to assure on schedule deliver of each iteration.

15. G2: for system testing

16. Use controllable factor to predict 16 Similar prediction model for system testing can be established. The possible impact factors could be: The defects injected in requirement, design and coding The defects removed respectively The defects removed in integrated testing of each iteration ……

17. Summary 17 All data can be get The factors which used to establish prediction model are controllable and independent Statistical techniques are used to do analysis Prediction object is relevant to the organization process improvement objectives The prediction model can be refined along with the using and more data was got

18. Thanks！ 18