SlideShare a Scribd company logo

Adequate and Precise Evaluation of Predictive Models in Software Engineering Studies

•Download as PPT, PDF•
•
Tim Menzies
Tim Menzies

Yan Ma and Bojan Cukic

Economy & FinanceTechnology
1 of 19
Yan Ma, Bojan Cukic Lane Department of Computer Science and Electrical Engineering West Virginia University May 2007 Adequate Evaluation of Quality Models in Software Engineering Studies CITeR The Center for Identification Technology Research www.citer.wvu.edu An NSF I/UCR Center advancing integrative research
Evaluating Defect Models ,[object Object],[object Object],[object Object],[object Object],[object Object]
Software defect data: Class Imbalance ,[object Object],[object Object]
Software Defect Data: Correlation MDP-PC1: Pearson correlation coefficients 1.000 0.473 0.982 0.987 0.996 0.924 N 0.473 1.000 0.468 0.468 0.464 0.545 LCC 0.982 0.468 1.000 0.995 0.971 0.931 B 0.987 0.468 0.995 1.000 0.976 0.937 V 0.996 0.464 0.971 0.976 1.000 0.908 TOpnd 0.924 0.545 0.931 0.937 0.908 1.000 LOC N LCC B V TOpnd LOC
Software Defect Data: Correlation (2) MDP-KC2: Pearson correlation coefficients 1.000 0.912 0.836 0.887 0.636 0.909 LOB 0.912 1.000 0.972 0.990 0.615 0.991 Top 0.836 0.972 1.000 0.970 0.577 0.968 IV.G 0.887 0.990 0.970 1.000 0.536 0.986 V 0.636 0.615 0.577 0.536 1.000 0.632 UOp 0.909 0.991 0.968 0.986 0.632 1.000 LOC LOB TOp IV.G V UOp LOC
Software Defect Data: Correlation (3) ,[object Object]
Software Defect Data: Module Size ,[object Object],[object Object],The 90 th percentile of LOC for the collection of defect modules and defect-free modules 131 165 114 167 99 Defect 55 72 47 55 42 Defect-free CM1 JM1 PC1 KC2 KC1
Software Defect Data: Close Neighbors ,[object Object],[object Object],97.96% 73.47% CM1 85.71% 75.32% PC1 75.46% 67.90% JM1 58.33% 58.33% KC2 73.62% 66.26% KC1 % of defect modules that has  2 among the three nearest neighbors in the majority class % of defect modules whose nearest neighbor is a majority class instance Project
Implications on Evaluation ,[object Object],[object Object],[object Object]
Classification Success Measures ,[object Object],[object Object],[object Object],[object Object],[object Object],Random Forests on PC1 (only 7% modules faulty )
Success Measures (2) ,[object Object],[object Object],The geometric mean of the two accuracies. Higher precision leads to “cheaper” V&V
Success Measures (3) ,[object Object],[object Object],[object Object],[object Object]
Comparing Models ,[object Object],[object Object],0.527 0.305 F-measure (  =2) 0.368 0.372 F-measure (  =1) 0.783 0.519 G-mean 2 0.426 0.399 G-mean 1 Figure 1. (b) Figure 1. (a) PC1: random forests at different voting cutoffs
Comparing Performance MDP-PC1 0.588 0.542 0.396 0.661 0.498 ED (  = 0.5) 0.200 0.264 0.436 0.077 0.290 F-measure (  = 2) 0.275 0.327 0.428 0.106 0.278 F-measure (  = 1) 0.410 0.480 0.649 0.253 0.529 G-mean 2 0.352 0.356 0.428 0.137 0.278 G-mean 1 0.938 0.933 0.918 0.924 0.892 Overall Accuracy 0.732 0.540 0.415 0.289 0.259 Precision 0.995 0.985 0.954 0.988 0.936 1 – PF 0.169 0.234 0.442 0.065 0.299 PD Bagging J48 IB1 Logistic Naïve Bayes Indices
Comparing Performance: KC-2 0.433 0.375 0.409 0.500 0.492 ED (  = 0.67) 0.377 0.329 0.361 0.435 0.427 ED (  = 0.5) 0.498 0.552 0.504 0.418 0.434 F-measure (  = 2) 0.543 0.562 0.496 0.472 0.501 F-measure (  = 1) 0.663 0.700 0.661 0.602 0.615 G-mean 2 0.549 0.562 0.496 0.483 0.518 G-mean 1 0.836 0.824 0.786 0.820 0.836 Overall Accuracy 0.639 0.578 0.483 0.599 0.674 Precision 0.931 0.896 0.858 0.932 0.950 1 - PF 0.472 0.546 0.509 0.389 0.398 PD Bagging J48 IB1 Logistic Naïve Bayes Indices
Visual Tools: Margin Plots
Visual Tools: ROC ,[object Object],[object Object],[object Object],Factor  depends of the misclassification cost.
Summary ,[object Object],[object Object],[object Object]
Current work ,[object Object],[object Object]

Recommended

COMPUTING THE GROWTH RATE OF STEM CELLS USING DIGITAL IMAGE PROCESSING
COMPUTING THE GROWTH RATE OF STEM CELLS USING DIGITAL IMAGE PROCESSING COMPUTING THE GROWTH RATE OF STEM CELLS USING DIGITAL IMAGE PROCESSING
COMPUTING THE GROWTH RATE OF STEM CELLS USING DIGITAL IMAGE PROCESSING Pratyusha Mahavadi
 
Comparison of optimization technique of power system stabilizer by using gea
Comparison of optimization technique of power system stabilizer by using geaComparison of optimization technique of power system stabilizer by using gea
Comparison of optimization technique of power system stabilizer by using geaIAEME Publication
 
samir_jain_UROP_poster_spr14
samir_jain_UROP_poster_spr14samir_jain_UROP_poster_spr14
samir_jain_UROP_poster_spr14Samir Jain
 
Software engineering lecture notes
Software engineering lecture notesSoftware engineering lecture notes
Software engineering lecture notesSiva Ayyakutti
 
FallThesisFinal
FallThesisFinalFallThesisFinal
FallThesisFinalLeora Juster
 
Personality prediction from network structure
Personality prediction from network structurePersonality prediction from network structure
Personality prediction from network structuremrcmilano
 
Kano GIS Day 2014 - The Application of Multivariate Geostatistical analyses i...
Kano GIS Day 2014 - The Application of Multivariate Geostatistical analyses i...Kano GIS Day 2014 - The Application of Multivariate Geostatistical analyses i...
Kano GIS Day 2014 - The Application of Multivariate Geostatistical analyses i...eHealth Africa
 
Comparing Machine Learning Algorithms in Text Mining
Comparing Machine Learning Algorithms in Text MiningComparing Machine Learning Algorithms in Text Mining
Comparing Machine Learning Algorithms in Text MiningAndrea Gigli
 

Recommended

COMPUTING THE GROWTH RATE OF STEM CELLS USING DIGITAL IMAGE PROCESSING
COMPUTING THE GROWTH RATE OF STEM CELLS USING DIGITAL IMAGE PROCESSING COMPUTING THE GROWTH RATE OF STEM CELLS USING DIGITAL IMAGE PROCESSING
COMPUTING THE GROWTH RATE OF STEM CELLS USING DIGITAL IMAGE PROCESSING Pratyusha Mahavadi
 
Comparison of optimization technique of power system stabilizer by using gea
Comparison of optimization technique of power system stabilizer by using geaComparison of optimization technique of power system stabilizer by using gea
Comparison of optimization technique of power system stabilizer by using geaIAEME Publication
 
samir_jain_UROP_poster_spr14
samir_jain_UROP_poster_spr14samir_jain_UROP_poster_spr14
samir_jain_UROP_poster_spr14Samir Jain
 
Software engineering lecture notes
Software engineering lecture notesSoftware engineering lecture notes
Software engineering lecture notesSiva Ayyakutti
 
FallThesisFinal
FallThesisFinalFallThesisFinal
FallThesisFinalLeora Juster
 
Personality prediction from network structure
Personality prediction from network structurePersonality prediction from network structure
Personality prediction from network structuremrcmilano
 
Kano GIS Day 2014 - The Application of Multivariate Geostatistical analyses i...
Kano GIS Day 2014 - The Application of Multivariate Geostatistical analyses i...Kano GIS Day 2014 - The Application of Multivariate Geostatistical analyses i...
Kano GIS Day 2014 - The Application of Multivariate Geostatistical analyses i...eHealth Africa
 
Comparing Machine Learning Algorithms in Text Mining
Comparing Machine Learning Algorithms in Text MiningComparing Machine Learning Algorithms in Text Mining
Comparing Machine Learning Algorithms in Text MiningAndrea Gigli
 
Prediction-based Model Selection in PLS-PM
Prediction-based Model Selection in PLS-PMPrediction-based Model Selection in PLS-PM
Prediction-based Model Selection in PLS-PMGalit Shmueli
 
Improving Hardware Efficiency for DNN Applications
Improving Hardware Efficiency for DNN ApplicationsImproving Hardware Efficiency for DNN Applications
Improving Hardware Efficiency for DNN ApplicationsChester Chen
 
IRJET - Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence M...
IRJET - Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence M...IRJET - Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence M...
IRJET - Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence M...IRJET Journal
 
IRJET- Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence Ma...
IRJET- Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence Ma...IRJET- Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence Ma...
IRJET- Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence Ma...IRJET Journal
 
Machine-learning scoring functions for molecular docking
Machine-learning scoring functions for molecular dockingMachine-learning scoring functions for molecular docking
Machine-learning scoring functions for molecular dockingPedro Ballester
 
Heterogeneous Defect Prediction (

ESEC/FSE 2015)
Heterogeneous Defect Prediction (

ESEC/FSE 2015)Heterogeneous Defect Prediction (

ESEC/FSE 2015)
Heterogeneous Defect Prediction (

ESEC/FSE 2015)Sung Kim
 
annInstance28Nov6pm
annInstance28Nov6pmannInstance28Nov6pm
annInstance28Nov6pmXunhu (Tiger) Sun
 
A Validation of Object-Oriented Design Metrics as Quality Indicators
A Validation of Object-Oriented Design Metrics as Quality IndicatorsA Validation of Object-Oriented Design Metrics as Quality Indicators
A Validation of Object-Oriented Design Metrics as Quality Indicatorsvie_dels
 
ASSESSMENT OF INTRICATE DG PLANNING WITH PRACTICAL LOAD MODELS BY USING PSO
ASSESSMENT OF INTRICATE DG PLANNING WITH PRACTICAL LOAD MODELS BY USING PSO ASSESSMENT OF INTRICATE DG PLANNING WITH PRACTICAL LOAD MODELS BY USING PSO
ASSESSMENT OF INTRICATE DG PLANNING WITH PRACTICAL LOAD MODELS BY USING PSO ecij
 
MultiModal Retrieval Image
MultiModal Retrieval ImageMultiModal Retrieval Image
MultiModal Retrieval ImageKonstantinos Zagoris
 
CDAC 2018 Pellegrini clustering ppi networks
CDAC 2018 Pellegrini clustering ppi networksCDAC 2018 Pellegrini clustering ppi networks
CDAC 2018 Pellegrini clustering ppi networksMarco Antoniotti
 
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)Fellowship at Vodafone FutureLab
 
Bayesian Autoencoders for anomaly detection in industrial environments
Bayesian Autoencoders for anomaly detection in industrial environmentsBayesian Autoencoders for anomaly detection in industrial environments
Bayesian Autoencoders for anomaly detection in industrial environmentsBang Xiang Yong
 
Promise 2011: "Local Bias and its Impacts on the Performance of Parametric Es...
Promise 2011: "Local Bias and its Impacts on the Performance of Parametric Es...Promise 2011: "Local Bias and its Impacts on the Performance of Parametric Es...
Promise 2011: "Local Bias and its Impacts on the Performance of Parametric Es...CS, NcState
 
Fuzzy Logic Based Decision System For PCB Defects Correction
Fuzzy Logic Based Decision System For PCB Defects CorrectionFuzzy Logic Based Decision System For PCB Defects Correction
Fuzzy Logic Based Decision System For PCB Defects CorrectionIJERA Editor
 
ADPD poster 2020 Cogstate in ADNI-3
ADPD poster 2020 Cogstate in ADNI-3ADPD poster 2020 Cogstate in ADNI-3
ADPD poster 2020 Cogstate in ADNI-3Chris Edgar
 
Pathogen Detection with Brewster's Angle Straddle Interferometry
Pathogen Detection with Brewster's Angle Straddle InterferometryPathogen Detection with Brewster's Angle Straddle Interferometry
Pathogen Detection with Brewster's Angle Straddle InterferometryLauren Brownlee
 
AN EFFICIENT FACE RECOGNITION EMPLOYING SVM AND BU-LDP
AN EFFICIENT FACE RECOGNITION EMPLOYING SVM AND BU-LDPAN EFFICIENT FACE RECOGNITION EMPLOYING SVM AND BU-LDP
AN EFFICIENT FACE RECOGNITION EMPLOYING SVM AND BU-LDPIRJET Journal
 
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...IJERD Editor
 
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)Fellowship at Vodafone FutureLab
 
Modeling the Effect of Size of Defect Proneness for Open-Source Software
Modeling the Effect of Size of Defect Proneness for Open-Source SoftwareModeling the Effect of Size of Defect Proneness for Open-Source Software
Modeling the Effect of Size of Defect Proneness for Open-Source SoftwareTim Menzies
 
Assessing the Reliability of a Human Estimator
Assessing the Reliability of a Human EstimatorAssessing the Reliability of a Human Estimator
Assessing the Reliability of a Human EstimatorTim Menzies
 

More Related Content

Similar to Adequate and Precise Evaluation of Predictive Models in Software Engineering Studies

Prediction-based Model Selection in PLS-PM
Prediction-based Model Selection in PLS-PMPrediction-based Model Selection in PLS-PM
Prediction-based Model Selection in PLS-PMGalit Shmueli
 
Improving Hardware Efficiency for DNN Applications
Improving Hardware Efficiency for DNN ApplicationsImproving Hardware Efficiency for DNN Applications
Improving Hardware Efficiency for DNN ApplicationsChester Chen
 
IRJET - Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence M...
IRJET - Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence M...IRJET - Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence M...
IRJET - Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence M...IRJET Journal
 
IRJET- Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence Ma...
IRJET- Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence Ma...IRJET- Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence Ma...
IRJET- Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence Ma...IRJET Journal
 
Machine-learning scoring functions for molecular docking
Machine-learning scoring functions for molecular dockingMachine-learning scoring functions for molecular docking
Machine-learning scoring functions for molecular dockingPedro Ballester
 
Heterogeneous Defect Prediction (

ESEC/FSE 2015)
Heterogeneous Defect Prediction (

ESEC/FSE 2015)Heterogeneous Defect Prediction (

ESEC/FSE 2015)
Heterogeneous Defect Prediction (

ESEC/FSE 2015)Sung Kim
 
A Validation of Object-Oriented Design Metrics as Quality Indicators
A Validation of Object-Oriented Design Metrics as Quality IndicatorsA Validation of Object-Oriented Design Metrics as Quality Indicators
A Validation of Object-Oriented Design Metrics as Quality Indicatorsvie_dels
 
ASSESSMENT OF INTRICATE DG PLANNING WITH PRACTICAL LOAD MODELS BY USING PSO
ASSESSMENT OF INTRICATE DG PLANNING WITH PRACTICAL LOAD MODELS BY USING PSO ASSESSMENT OF INTRICATE DG PLANNING WITH PRACTICAL LOAD MODELS BY USING PSO
ASSESSMENT OF INTRICATE DG PLANNING WITH PRACTICAL LOAD MODELS BY USING PSO ecij
 
CDAC 2018 Pellegrini clustering ppi networks
CDAC 2018 Pellegrini clustering ppi networksCDAC 2018 Pellegrini clustering ppi networks
CDAC 2018 Pellegrini clustering ppi networksMarco Antoniotti
 
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)Fellowship at Vodafone FutureLab
 
Bayesian Autoencoders for anomaly detection in industrial environments
Bayesian Autoencoders for anomaly detection in industrial environmentsBayesian Autoencoders for anomaly detection in industrial environments
Bayesian Autoencoders for anomaly detection in industrial environmentsBang Xiang Yong
 
Promise 2011: "Local Bias and its Impacts on the Performance of Parametric Es...
Promise 2011: "Local Bias and its Impacts on the Performance of Parametric Es...Promise 2011: "Local Bias and its Impacts on the Performance of Parametric Es...
Promise 2011: "Local Bias and its Impacts on the Performance of Parametric Es...CS, NcState
 
Fuzzy Logic Based Decision System For PCB Defects Correction
Fuzzy Logic Based Decision System For PCB Defects CorrectionFuzzy Logic Based Decision System For PCB Defects Correction
Fuzzy Logic Based Decision System For PCB Defects CorrectionIJERA Editor
 
ADPD poster 2020 Cogstate in ADNI-3
ADPD poster 2020 Cogstate in ADNI-3ADPD poster 2020 Cogstate in ADNI-3
ADPD poster 2020 Cogstate in ADNI-3Chris Edgar
 
Pathogen Detection with Brewster's Angle Straddle Interferometry
Pathogen Detection with Brewster's Angle Straddle InterferometryPathogen Detection with Brewster's Angle Straddle Interferometry
Pathogen Detection with Brewster's Angle Straddle InterferometryLauren Brownlee
 
AN EFFICIENT FACE RECOGNITION EMPLOYING SVM AND BU-LDP
AN EFFICIENT FACE RECOGNITION EMPLOYING SVM AND BU-LDPAN EFFICIENT FACE RECOGNITION EMPLOYING SVM AND BU-LDP
AN EFFICIENT FACE RECOGNITION EMPLOYING SVM AND BU-LDPIRJET Journal
 
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...IJERD Editor
 
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)Fellowship at Vodafone FutureLab
 

Similar to Adequate and Precise Evaluation of Predictive Models in Software Engineering Studies (20)

Prediction-based Model Selection in PLS-PM
Prediction-based Model Selection in PLS-PMPrediction-based Model Selection in PLS-PM
Prediction-based Model Selection in PLS-PM
 
Improving Hardware Efficiency for DNN Applications
Improving Hardware Efficiency for DNN ApplicationsImproving Hardware Efficiency for DNN Applications
Improving Hardware Efficiency for DNN Applications
 
IRJET - Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence M...
IRJET - Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence M...IRJET - Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence M...
IRJET - Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence M...
 
IRJET- Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence Ma...
IRJET- Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence Ma...IRJET- Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence Ma...
IRJET- Plant Leaf Disease Diagnosis from Color Imagery using Co-Occurrence Ma...
 
Machine-learning scoring functions for molecular docking
Machine-learning scoring functions for molecular dockingMachine-learning scoring functions for molecular docking
Machine-learning scoring functions for molecular docking
 
Heterogeneous Defect Prediction (

ESEC/FSE 2015)
Heterogeneous Defect Prediction (

ESEC/FSE 2015)Heterogeneous Defect Prediction (

ESEC/FSE 2015)
Heterogeneous Defect Prediction (

ESEC/FSE 2015)
 
annInstance28Nov6pm
annInstance28Nov6pmannInstance28Nov6pm
annInstance28Nov6pm
 
A Validation of Object-Oriented Design Metrics as Quality Indicators
A Validation of Object-Oriented Design Metrics as Quality IndicatorsA Validation of Object-Oriented Design Metrics as Quality Indicators
A Validation of Object-Oriented Design Metrics as Quality Indicators
 
ASSESSMENT OF INTRICATE DG PLANNING WITH PRACTICAL LOAD MODELS BY USING PSO
ASSESSMENT OF INTRICATE DG PLANNING WITH PRACTICAL LOAD MODELS BY USING PSO ASSESSMENT OF INTRICATE DG PLANNING WITH PRACTICAL LOAD MODELS BY USING PSO
ASSESSMENT OF INTRICATE DG PLANNING WITH PRACTICAL LOAD MODELS BY USING PSO
 
MultiModal Retrieval Image
MultiModal Retrieval ImageMultiModal Retrieval Image
MultiModal Retrieval Image
 
CDAC 2018 Pellegrini clustering ppi networks
CDAC 2018 Pellegrini clustering ppi networksCDAC 2018 Pellegrini clustering ppi networks
CDAC 2018 Pellegrini clustering ppi networks
 
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
 
Bayesian Autoencoders for anomaly detection in industrial environments
Bayesian Autoencoders for anomaly detection in industrial environmentsBayesian Autoencoders for anomaly detection in industrial environments
Bayesian Autoencoders for anomaly detection in industrial environments
 
Promise 2011: "Local Bias and its Impacts on the Performance of Parametric Es...
Promise 2011: "Local Bias and its Impacts on the Performance of Parametric Es...Promise 2011: "Local Bias and its Impacts on the Performance of Parametric Es...
Promise 2011: "Local Bias and its Impacts on the Performance of Parametric Es...
 
Fuzzy Logic Based Decision System For PCB Defects Correction
Fuzzy Logic Based Decision System For PCB Defects CorrectionFuzzy Logic Based Decision System For PCB Defects Correction
Fuzzy Logic Based Decision System For PCB Defects Correction
 
ADPD poster 2020 Cogstate in ADNI-3
ADPD poster 2020 Cogstate in ADNI-3ADPD poster 2020 Cogstate in ADNI-3
ADPD poster 2020 Cogstate in ADNI-3
 
Pathogen Detection with Brewster's Angle Straddle Interferometry
Pathogen Detection with Brewster's Angle Straddle InterferometryPathogen Detection with Brewster's Angle Straddle Interferometry
Pathogen Detection with Brewster's Angle Straddle Interferometry
 
AN EFFICIENT FACE RECOGNITION EMPLOYING SVM AND BU-LDP
AN EFFICIENT FACE RECOGNITION EMPLOYING SVM AND BU-LDPAN EFFICIENT FACE RECOGNITION EMPLOYING SVM AND BU-LDP
AN EFFICIENT FACE RECOGNITION EMPLOYING SVM AND BU-LDP
 
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...
 
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
AlexNet(ImageNet Classification with Deep Convolutional Neural Networks)
 

More from Tim Menzies

Modeling the Effect of Size of Defect Proneness for Open-Source Software
Modeling the Effect of Size of Defect Proneness for Open-Source SoftwareModeling the Effect of Size of Defect Proneness for Open-Source Software
Modeling the Effect of Size of Defect Proneness for Open-Source SoftwareTim Menzies
 
Assessing the Reliability of a Human Estimator
Assessing the Reliability of a Human EstimatorAssessing the Reliability of a Human Estimator
Assessing the Reliability of a Human EstimatorTim Menzies
 
Experiments on Design Pattern Discovery
Experiments on Design Pattern DiscoveryExperiments on Design Pattern Discovery
Experiments on Design Pattern DiscoveryTim Menzies
 
Using Developer Information as a Prediction Factor
Using Developer Information as a Prediction FactorUsing Developer Information as a Prediction Factor
Using Developer Information as a Prediction FactorTim Menzies
 
Project Data Incorporating Qualitative Factors for Improved Software Defect P...
Project Data Incorporating Qualitative Factors for Improved Software Defect P...Project Data Incorporating Qualitative Factors for Improved Software Defect P...
Project Data Incorporating Qualitative Factors for Improved Software Defect P...Tim Menzies
 
Make the Most of Your Time: How Should the Analyst Work with Automated Tracea...
Make the Most of Your Time: How Should the Analyst Work with Automated Tracea...Make the Most of Your Time: How Should the Analyst Work with Automated Tracea...
Make the Most of Your Time: How Should the Analyst Work with Automated Tracea...Tim Menzies
 
Complexity Measures for Secure Service-Orieted Software Architectures
Complexity Measures for Secure Service-Orieted Software ArchitecturesComplexity Measures for Secure Service-Orieted Software Architectures
Complexity Measures for Secure Service-Orieted Software ArchitecturesTim Menzies
 
Decision Support Analyss for Software Effort Estimation by Analogy
Decision Support Analyss for Software Effort Estimation by AnalogyDecision Support Analyss for Software Effort Estimation by Analogy
Decision Support Analyss for Software Effort Estimation by AnalogyTim Menzies
 
Predicting Defects for Eclipse
Predicting Defects for EclipsePredicting Defects for Eclipse
Predicting Defects for EclipseTim Menzies
 

More from Tim Menzies (10)

Modeling the Effect of Size of Defect Proneness for Open-Source Software
Modeling the Effect of Size of Defect Proneness for Open-Source SoftwareModeling the Effect of Size of Defect Proneness for Open-Source Software
Modeling the Effect of Size of Defect Proneness for Open-Source Software
 
Assessing the Reliability of a Human Estimator
Assessing the Reliability of a Human EstimatorAssessing the Reliability of a Human Estimator
Assessing the Reliability of a Human Estimator
 
Experiments on Design Pattern Discovery
Experiments on Design Pattern DiscoveryExperiments on Design Pattern Discovery
Experiments on Design Pattern Discovery
 
Using Developer Information as a Prediction Factor
Using Developer Information as a Prediction FactorUsing Developer Information as a Prediction Factor
Using Developer Information as a Prediction Factor
 
Project Data Incorporating Qualitative Factors for Improved Software Defect P...
Project Data Incorporating Qualitative Factors for Improved Software Defect P...Project Data Incorporating Qualitative Factors for Improved Software Defect P...
Project Data Incorporating Qualitative Factors for Improved Software Defect P...
 
Make the Most of Your Time: How Should the Analyst Work with Automated Tracea...
Make the Most of Your Time: How Should the Analyst Work with Automated Tracea...Make the Most of Your Time: How Should the Analyst Work with Automated Tracea...
Make the Most of Your Time: How Should the Analyst Work with Automated Tracea...
 
Promise Keynote
Promise KeynotePromise Keynote
Promise Keynote
 
Complexity Measures for Secure Service-Orieted Software Architectures
Complexity Measures for Secure Service-Orieted Software ArchitecturesComplexity Measures for Secure Service-Orieted Software Architectures
Complexity Measures for Secure Service-Orieted Software Architectures
 
Decision Support Analyss for Software Effort Estimation by Analogy
Decision Support Analyss for Software Effort Estimation by AnalogyDecision Support Analyss for Software Effort Estimation by Analogy
Decision Support Analyss for Software Effort Estimation by Analogy
 
Predicting Defects for Eclipse
Predicting Defects for EclipsePredicting Defects for Eclipse
Predicting Defects for Eclipse
 

Recently uploaded

Basic concepts related to Financial modelling
Basic concepts related to Financial modellingBasic concepts related to Financial modelling
Basic concepts related to Financial modellingbaijup5
 
(DIYA) Bhumkar Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(DIYA) Bhumkar Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...(DIYA) Bhumkar Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(DIYA) Bhumkar Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...ranjana rawat
 
Top Rated Pune Call Girls Viman Nagar ⟟ 6297143586 ⟟ Call Me For Genuine Sex...
Top Rated Pune Call Girls Viman Nagar ⟟ 6297143586 ⟟ Call Me For Genuine Sex...Top Rated Pune Call Girls Viman Nagar ⟟ 6297143586 ⟟ Call Me For Genuine Sex...
Top Rated Pune Call Girls Viman Nagar ⟟ 6297143586 ⟟ Call Me For Genuine Sex...Call Girls in Nagpur High Profile
 
Call US 📞 9892124323 ✅ Kurla Call Girls In Kurla ( Mumbai ) secure service
Call US 📞 9892124323 ✅ Kurla Call Girls In Kurla ( Mumbai ) secure serviceCall US 📞 9892124323 ✅ Kurla Call Girls In Kurla ( Mumbai ) secure service
Call US 📞 9892124323 ✅ Kurla Call Girls In Kurla ( Mumbai ) secure servicePooja Nehwal
 
The Economic History of the U.S. Lecture 19.pdf
The Economic History of the U.S. Lecture 19.pdfThe Economic History of the U.S. Lecture 19.pdf
The Economic History of the U.S. Lecture 19.pdfGale Pooley
 
Solution Manual for Financial Accounting, 11th Edition by Robert Libby, Patri...
Solution Manual for Financial Accounting, 11th Edition by Robert Libby, Patri...Solution Manual for Financial Accounting, 11th Edition by Robert Libby, Patri...
Solution Manual for Financial Accounting, 11th Edition by Robert Libby, Patri...ssifa0344
 
The Economic History of the U.S. Lecture 25.pdf
The Economic History of the U.S. Lecture 25.pdfThe Economic History of the U.S. Lecture 25.pdf
The Economic History of the U.S. Lecture 25.pdfGale Pooley
 
The Economic History of the U.S. Lecture 17.pdf
The Economic History of the U.S. Lecture 17.pdfThe Economic History of the U.S. Lecture 17.pdf
The Economic History of the U.S. Lecture 17.pdfGale Pooley
 
The Economic History of the U.S. Lecture 30.pdf
The Economic History of the U.S. Lecture 30.pdfThe Economic History of the U.S. Lecture 30.pdf
The Economic History of the U.S. Lecture 30.pdfGale Pooley
 
The Economic History of the U.S. Lecture 18.pdf
The Economic History of the U.S. Lecture 18.pdfThe Economic History of the U.S. Lecture 18.pdf
The Economic History of the U.S. Lecture 18.pdfGale Pooley
 
(ANIKA) Budhwar Peth Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANIKA) Budhwar Peth Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...(ANIKA) Budhwar Peth Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANIKA) Budhwar Peth Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...ranjana rawat
 
Booking open Available Pune Call Girls Shivane 6297143586 Call Hot Indian Gi...
Booking open Available Pune Call Girls Shivane 6297143586 Call Hot Indian Gi...Booking open Available Pune Call Girls Shivane 6297143586 Call Hot Indian Gi...
Booking open Available Pune Call Girls Shivane 6297143586 Call Hot Indian Gi...Call Girls in Nagpur High Profile
 
The Economic History of the U.S. Lecture 23.pdf
The Economic History of the U.S. Lecture 23.pdfThe Economic History of the U.S. Lecture 23.pdf
The Economic History of the U.S. Lecture 23.pdfGale Pooley
 
Booking open Available Pune Call Girls Talegaon Dabhade 6297143586 Call Hot ...
Booking open Available Pune Call Girls Talegaon Dabhade 6297143586 Call Hot ...Booking open Available Pune Call Girls Talegaon Dabhade 6297143586 Call Hot ...
Booking open Available Pune Call Girls Talegaon Dabhade 6297143586 Call Hot ...Call Girls in Nagpur High Profile
 
VVIP Pune Call Girls Katraj (7001035870) Pune Escorts Nearby with Complete Sa...
VVIP Pune Call Girls Katraj (7001035870) Pune Escorts Nearby with Complete Sa...VVIP Pune Call Girls Katraj (7001035870) Pune Escorts Nearby with Complete Sa...
VVIP Pune Call Girls Katraj (7001035870) Pune Escorts Nearby with Complete Sa...Call Girls in Nagpur High Profile
 
WhatsApp 📞 Call : 9892124323 ✅Call Girls In Chembur ( Mumbai ) secure service
WhatsApp 📞 Call : 9892124323 ✅Call Girls In Chembur ( Mumbai ) secure serviceWhatsApp 📞 Call : 9892124323 ✅Call Girls In Chembur ( Mumbai ) secure service
WhatsApp 📞 Call : 9892124323 ✅Call Girls In Chembur ( Mumbai ) secure servicePooja Nehwal
 
VIP Independent Call Girls in Andheri 🌹 9920725232 ( Call Me ) Mumbai Escorts...
VIP Independent Call Girls in Andheri 🌹 9920725232 ( Call Me ) Mumbai Escorts...VIP Independent Call Girls in Andheri 🌹 9920725232 ( Call Me ) Mumbai Escorts...
VIP Independent Call Girls in Andheri 🌹 9920725232 ( Call Me ) Mumbai Escorts...dipikadinghjn ( Why You Choose Us? ) Escorts
 
The Economic History of the U.S. Lecture 21.pdf
The Economic History of the U.S. Lecture 21.pdfThe Economic History of the U.S. Lecture 21.pdf
The Economic History of the U.S. Lecture 21.pdfGale Pooley
 

Recently uploaded (20)

Basic concepts related to Financial modelling
Basic concepts related to Financial modellingBasic concepts related to Financial modelling
Basic concepts related to Financial modelling
 
(DIYA) Bhumkar Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(DIYA) Bhumkar Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...(DIYA) Bhumkar Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(DIYA) Bhumkar Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
 
Top Rated Pune Call Girls Viman Nagar ⟟ 6297143586 ⟟ Call Me For Genuine Sex...
Top Rated Pune Call Girls Viman Nagar ⟟ 6297143586 ⟟ Call Me For Genuine Sex...Top Rated Pune Call Girls Viman Nagar ⟟ 6297143586 ⟟ Call Me For Genuine Sex...
Top Rated Pune Call Girls Viman Nagar ⟟ 6297143586 ⟟ Call Me For Genuine Sex...
 
Call US 📞 9892124323 ✅ Kurla Call Girls In Kurla ( Mumbai ) secure service
Call US 📞 9892124323 ✅ Kurla Call Girls In Kurla ( Mumbai ) secure serviceCall US 📞 9892124323 ✅ Kurla Call Girls In Kurla ( Mumbai ) secure service
Call US 📞 9892124323 ✅ Kurla Call Girls In Kurla ( Mumbai ) secure service
 
The Economic History of the U.S. Lecture 19.pdf
The Economic History of the U.S. Lecture 19.pdfThe Economic History of the U.S. Lecture 19.pdf
The Economic History of the U.S. Lecture 19.pdf
 
Solution Manual for Financial Accounting, 11th Edition by Robert Libby, Patri...
Solution Manual for Financial Accounting, 11th Edition by Robert Libby, Patri...Solution Manual for Financial Accounting, 11th Edition by Robert Libby, Patri...
Solution Manual for Financial Accounting, 11th Edition by Robert Libby, Patri...
 
The Economic History of the U.S. Lecture 25.pdf
The Economic History of the U.S. Lecture 25.pdfThe Economic History of the U.S. Lecture 25.pdf
The Economic History of the U.S. Lecture 25.pdf
 
The Economic History of the U.S. Lecture 17.pdf
The Economic History of the U.S. Lecture 17.pdfThe Economic History of the U.S. Lecture 17.pdf
The Economic History of the U.S. Lecture 17.pdf
 
The Economic History of the U.S. Lecture 30.pdf
The Economic History of the U.S. Lecture 30.pdfThe Economic History of the U.S. Lecture 30.pdf
The Economic History of the U.S. Lecture 30.pdf
 
The Economic History of the U.S. Lecture 18.pdf
The Economic History of the U.S. Lecture 18.pdfThe Economic History of the U.S. Lecture 18.pdf
The Economic History of the U.S. Lecture 18.pdf
 
(ANIKA) Budhwar Peth Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANIKA) Budhwar Peth Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...(ANIKA) Budhwar Peth Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANIKA) Budhwar Peth Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
 
Booking open Available Pune Call Girls Shivane 6297143586 Call Hot Indian Gi...
Booking open Available Pune Call Girls Shivane 6297143586 Call Hot Indian Gi...Booking open Available Pune Call Girls Shivane 6297143586 Call Hot Indian Gi...
Booking open Available Pune Call Girls Shivane 6297143586 Call Hot Indian Gi...
 
The Economic History of the U.S. Lecture 23.pdf
The Economic History of the U.S. Lecture 23.pdfThe Economic History of the U.S. Lecture 23.pdf
The Economic History of the U.S. Lecture 23.pdf
 
Booking open Available Pune Call Girls Talegaon Dabhade 6297143586 Call Hot ...
Booking open Available Pune Call Girls Talegaon Dabhade 6297143586 Call Hot ...Booking open Available Pune Call Girls Talegaon Dabhade 6297143586 Call Hot ...
Booking open Available Pune Call Girls Talegaon Dabhade 6297143586 Call Hot ...
 
VVIP Pune Call Girls Katraj (7001035870) Pune Escorts Nearby with Complete Sa...
VVIP Pune Call Girls Katraj (7001035870) Pune Escorts Nearby with Complete Sa...VVIP Pune Call Girls Katraj (7001035870) Pune Escorts Nearby with Complete Sa...
VVIP Pune Call Girls Katraj (7001035870) Pune Escorts Nearby with Complete Sa...
 
WhatsApp 📞 Call : 9892124323 ✅Call Girls In Chembur ( Mumbai ) secure service
WhatsApp 📞 Call : 9892124323 ✅Call Girls In Chembur ( Mumbai ) secure serviceWhatsApp 📞 Call : 9892124323 ✅Call Girls In Chembur ( Mumbai ) secure service
WhatsApp 📞 Call : 9892124323 ✅Call Girls In Chembur ( Mumbai ) secure service
 
VIP Independent Call Girls in Andheri 🌹 9920725232 ( Call Me ) Mumbai Escorts...
VIP Independent Call Girls in Andheri 🌹 9920725232 ( Call Me ) Mumbai Escorts...VIP Independent Call Girls in Andheri 🌹 9920725232 ( Call Me ) Mumbai Escorts...
VIP Independent Call Girls in Andheri 🌹 9920725232 ( Call Me ) Mumbai Escorts...
 
Veritas Interim Report 1 January–31 March 2024
Veritas Interim Report 1 January–31 March 2024Veritas Interim Report 1 January–31 March 2024
Veritas Interim Report 1 January–31 March 2024
 
The Economic History of the U.S. Lecture 21.pdf
The Economic History of the U.S. Lecture 21.pdfThe Economic History of the U.S. Lecture 21.pdf
The Economic History of the U.S. Lecture 21.pdf
 
(INDIRA) Call Girl Mumbai Call Now 8250077686 Mumbai Escorts 24x7
(INDIRA) Call Girl Mumbai Call Now 8250077686 Mumbai Escorts 24x7(INDIRA) Call Girl Mumbai Call Now 8250077686 Mumbai Escorts 24x7
(INDIRA) Call Girl Mumbai Call Now 8250077686 Mumbai Escorts 24x7
 

Adequate and Precise Evaluation of Predictive Models in Software Engineering Studies

  • 1. Yan Ma, Bojan Cukic Lane Department of Computer Science and Electrical Engineering West Virginia University May 2007 Adequate Evaluation of Quality Models in Software Engineering Studies CITeR The Center for Identification Technology Research www.citer.wvu.edu An NSF I/UCR Center advancing integrative research
  • 2.
  • 3.
  • 4. Software Defect Data: Correlation MDP-PC1: Pearson correlation coefficients 1.000 0.473 0.982 0.987 0.996 0.924 N 0.473 1.000 0.468 0.468 0.464 0.545 LCC 0.982 0.468 1.000 0.995 0.971 0.931 B 0.987 0.468 0.995 1.000 0.976 0.937 V 0.996 0.464 0.971 0.976 1.000 0.908 TOpnd 0.924 0.545 0.931 0.937 0.908 1.000 LOC N LCC B V TOpnd LOC
  • 5. Software Defect Data: Correlation (2) MDP-KC2: Pearson correlation coefficients 1.000 0.912 0.836 0.887 0.636 0.909 LOB 0.912 1.000 0.972 0.990 0.615 0.991 Top 0.836 0.972 1.000 0.970 0.577 0.968 IV.G 0.887 0.990 0.970 1.000 0.536 0.986 V 0.636 0.615 0.577 0.536 1.000 0.632 UOp 0.909 0.991 0.968 0.986 0.632 1.000 LOC LOB TOp IV.G V UOp LOC
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14. Comparing Performance MDP-PC1 0.588 0.542 0.396 0.661 0.498 ED (  = 0.5) 0.200 0.264 0.436 0.077 0.290 F-measure (  = 2) 0.275 0.327 0.428 0.106 0.278 F-measure (  = 1) 0.410 0.480 0.649 0.253 0.529 G-mean 2 0.352 0.356 0.428 0.137 0.278 G-mean 1 0.938 0.933 0.918 0.924 0.892 Overall Accuracy 0.732 0.540 0.415 0.289 0.259 Precision 0.995 0.985 0.954 0.988 0.936 1 – PF 0.169 0.234 0.442 0.065 0.299 PD Bagging J48 IB1 Logistic Naïve Bayes Indices
  • 15. Comparing Performance: KC-2 0.433 0.375 0.409 0.500 0.492 ED (  = 0.67) 0.377 0.329 0.361 0.435 0.427 ED (  = 0.5) 0.498 0.552 0.504 0.418 0.434 F-measure (  = 2) 0.543 0.562 0.496 0.472 0.501 F-measure (  = 1) 0.663 0.700 0.661 0.602 0.615 G-mean 2 0.549 0.562 0.496 0.483 0.518 G-mean 1 0.836 0.824 0.786 0.820 0.836 Overall Accuracy 0.639 0.578 0.483 0.599 0.674 Precision 0.931 0.896 0.858 0.932 0.950 1 - PF 0.472 0.546 0.509 0.389 0.398 PD Bagging J48 IB1 Logistic Naïve Bayes Indices
  • 17.
  • 18.
  • 19.