SlideShare uma empresa Scribd logo
1 de 22
Baixar para ler offline
HowHighWillItBe?
Using Machine
Learning Modelsto
PredictBranch
Coverage in
AutomatedTesting
G. Grano,T.Titov, S. Panichella, H. Gall
MaLTeSQuE@SANER 2018, 20, Campobasso (Italy)
grano@ifi.uzh.ch giograno90
U.S. EconomyImpact1
$59.5 billion
$22.2 billion
avoidable
0.6% GDP
1
http://www.abeacha.com/NIST_press_release_bugs_cost.html
2 — Giovanni Grano @ s.e.a.l.
Continuous
Integration
The way we develop and release code is
rapidly changing
In an ideal world, at every single commit the
entire test suite should be executed ...
... but companies like Google commit about
16.000 changes per day!
3 — Giovanni Grano @ s.e.a.l.
TestData
Generation
Pretty active research area in the last
years
Mature tools able to generate test suites
with high coverage:
> EvoSuite
> Randoop
> ...
> and many more!
How do they fit in such a CI/CD environment?
4 — Giovanni Grano @ s.e.a.l.
Continuous
Testing
Generation 2
Continuous integration enhanced with
automated test generation
It raises many questions:
> testing order
> how much time to spend per class
2
Campos et al - Continuous Test Generation: Enhancing Continuous Integration with
Automated Test Generation
5 — Giovanni Grano @ s.e.a.l.
TestSuite
Augmentatio
n 3
Automatic generation considering code
changes and their effect on the previous
codebase
Hardy doable to the expensive amount of
time needed to generate tests
3
Xu et al - Directed Test Suite Augmentation: Techniques and Tradeoffs
6 — Giovanni Grano @ s.e.a.l.
Coverage
Prediction
Knowing a priori the coverage achieved by
test data generation tools
> maximize the coverage for the entire
system given an amount of time
> budget allocation for critical
components
7 — Giovanni Grano @ s.e.a.l.
Research Questions
> RQ1: Which type of features can we leverage to train machine
learning models to predict the branch coverage achieved by
test data generation tools?
> RQ2: To what extend can we predict the coverage achieved by
test data generation tools?
9 — Giovanni Grano @ s.e.a.l.
Project
Selection
Open Source Projects from Defect4j
> Apache Cassandra
> Apache Ivy
> Google Guava
> Google Dagger
Guava Cassandr
a
Dagger Ivy
LOC 78,525 220,573 848 50,430
Java
Files
538 1,474 43 464
10 — Giovanni Grano @ s.e.a.l.
Metrics Extractor
Coverage
Training Set
> Run EvoSuite
> Labeled data (with coverage)
> Computed metrics = input variables
Threats:
> we did it once
11 — Giovanni Grano @ s.e.a.l.
Feature
Selection
Goal: capture
complexity
Package LevelFeatures
Computed with JDepend 4
Name Description
Ca indicator of the package's responsibility
Ce indicator of the package's independence
A abstract classes / total number of classes
I indicator of the package's resilience to change
... ...
4
https://github.com/clarkware/jdepend
13 — Giovanni Grano @ s.e.a.l.
CKand OO Feature
ck tool provided by Aniche 5
Name Description
CBO coupling between objects
DIT depth of inheritance tree
NOC number of children
NOSF number of static field
... ...
5
https://github.com/mauricioaniche/ck
14 — Giovanni Grano @ s.e.a.l.
Java
Reserved
Keywords
To capture additional complexity
Previously used in Information Retrieval as a
feature 6
52 Java reserved keywords
6
Sanderson et al The history of information retrieval research
15 — Giovanni Grano @ s.e.a.l.
Grid Selection
best hyper-parameters
3-cross fold validation
FeatureTransformation
z-score
16 — Giovanni Grano @ s.e.a.l.
Algorithms
Huber Regression
SupportVector
Regression
Multi-Layer Perception
17 — Giovanni Grano @ s.e.a.l.
Results RQ1
Huber SVR MLP Tool's
Average
EvoSuit
e
0.255 0.216 0.242 0.238
Randoop 0.172 0.088 0.139 0.131
Average 0.213 0.152 0.191
18 — Giovanni Grano @ s.e.a.l.
Results RQ2
Time Math Lang Tool's
Aver.
EvoSuit
e
0.255 0.330 0.289 0.291
Randoo
p
0.168 0.262 0.246 0.225
Aver. 0.211 0.296 0.267
19 — Giovanni Grano @ s.e.a.l.
Conclusion
Knowing a priori the coverage achieved by
test data generation tools might ease
important decisions;
> we took the first steps, investigating
well known features
> well know features (from gut feeling)
give reasonable results
20 — Giovanni Grano @ s.e.a.l.
Futurework
Improvements
> larger dataset
> branch-level features
> feature analysis
> differences between employed tools
21 — Giovanni Grano @ s.e.a.l.
How High Will It Be? Using Machine Learning Models to Predict Branch Coverage in Automated Testing

Mais conteúdo relacionado

Semelhante a How High Will It Be? Using Machine Learning Models to Predict Branch Coverage in Automated Testing

jRecruiter - The AJUG Job Posting Service
jRecruiter - The AJUG Job Posting ServicejRecruiter - The AJUG Job Posting Service
jRecruiter - The AJUG Job Posting ServiceGunnar Hillert
 
Easy path to machine learning (Spring 2020)
Easy path to machine learning (Spring 2020)Easy path to machine learning (Spring 2020)
Easy path to machine learning (Spring 2020)wesley chun
 
OSMC 2022 | Unifying Observability Weaving Prometheus, Jaeger, and Open Sourc...
OSMC 2022 | Unifying Observability Weaving Prometheus, Jaeger, and Open Sourc...OSMC 2022 | Unifying Observability Weaving Prometheus, Jaeger, and Open Sourc...
OSMC 2022 | Unifying Observability Weaving Prometheus, Jaeger, and Open Sourc...NETWAYS
 
20090918 Agile Computer Control of a Complex Experiment
20090918 Agile Computer Control of a Complex Experiment20090918 Agile Computer Control of a Complex Experiment
20090918 Agile Computer Control of a Complex ExperimentJonathan Blakes
 
building intelligent systems with large scale deep learning
building intelligent systems with large scale deep learningbuilding intelligent systems with large scale deep learning
building intelligent systems with large scale deep learningmustafa sarac
 
Exploring the Integration of User Feedback in Automated Testing of Android Ap...
Exploring the Integration of User Feedback in Automated Testing of Android Ap...Exploring the Integration of User Feedback in Automated Testing of Android Ap...
Exploring the Integration of User Feedback in Automated Testing of Android Ap...Sebastiano Panichella
 
Test-Driven Design Insights@DevoxxBE 2023.pptx
Test-Driven Design Insights@DevoxxBE 2023.pptxTest-Driven Design Insights@DevoxxBE 2023.pptx
Test-Driven Design Insights@DevoxxBE 2023.pptxVictor Rentea
 
Continuous testing - GUERLAIS ARGOT - Air France KLM Sogeti- Soirée du Test L...
Continuous testing - GUERLAIS ARGOT - Air France KLM Sogeti- Soirée du Test L...Continuous testing - GUERLAIS ARGOT - Air France KLM Sogeti- Soirée du Test L...
Continuous testing - GUERLAIS ARGOT - Air France KLM Sogeti- Soirée du Test L...TelecomValley
 
Keynote WFIoT2019 - Data Graph, Knowledge Graphs Ontologies, Internet of Thin...
Keynote WFIoT2019 - Data Graph, Knowledge Graphs Ontologies, Internet of Thin...Keynote WFIoT2019 - Data Graph, Knowledge Graphs Ontologies, Internet of Thin...
Keynote WFIoT2019 - Data Graph, Knowledge Graphs Ontologies, Internet of Thin...Amélie Gyrard
 
Adtech scala-performance-tuning-150323223738-conversion-gate01
Adtech scala-performance-tuning-150323223738-conversion-gate01Adtech scala-performance-tuning-150323223738-conversion-gate01
Adtech scala-performance-tuning-150323223738-conversion-gate01Giridhar Addepalli
 
Adtech x Scala x Performance tuning
Adtech x Scala x Performance tuningAdtech x Scala x Performance tuning
Adtech x Scala x Performance tuningYosuke Mizutani
 
Measuring Technical Lag in Software Deployments (CHAOSScon 2020)
Measuring Technical Lag in Software Deployments (CHAOSScon 2020)Measuring Technical Lag in Software Deployments (CHAOSScon 2020)
Measuring Technical Lag in Software Deployments (CHAOSScon 2020)Tom Mens
 
Quality of life through Unit Testing
Quality of life through Unit TestingQuality of life through Unit Testing
Quality of life through Unit TestingSian Lerk Lau
 
Devoxx 2014 [incomplete] summary
Devoxx 2014 [incomplete] summaryDevoxx 2014 [incomplete] summary
Devoxx 2014 [incomplete] summaryArtem Oboturov
 
Grokking Techtalk #38: Escape Analysis in Go compiler
 Grokking Techtalk #38: Escape Analysis in Go compiler Grokking Techtalk #38: Escape Analysis in Go compiler
Grokking Techtalk #38: Escape Analysis in Go compilerGrokking VN
 
Csw2016 d antoine_automatic_exploitgeneration
Csw2016 d antoine_automatic_exploitgenerationCsw2016 d antoine_automatic_exploitgeneration
Csw2016 d antoine_automatic_exploitgenerationCanSecWest
 

Semelhante a How High Will It Be? Using Machine Learning Models to Predict Branch Coverage in Automated Testing (20)

jRecruiter - The AJUG Job Posting Service
jRecruiter - The AJUG Job Posting ServicejRecruiter - The AJUG Job Posting Service
jRecruiter - The AJUG Job Posting Service
 
Easy path to machine learning (Spring 2020)
Easy path to machine learning (Spring 2020)Easy path to machine learning (Spring 2020)
Easy path to machine learning (Spring 2020)
 
OSMC 2022 | Unifying Observability Weaving Prometheus, Jaeger, and Open Sourc...
OSMC 2022 | Unifying Observability Weaving Prometheus, Jaeger, and Open Sourc...OSMC 2022 | Unifying Observability Weaving Prometheus, Jaeger, and Open Sourc...
OSMC 2022 | Unifying Observability Weaving Prometheus, Jaeger, and Open Sourc...
 
20090918 Agile Computer Control of a Complex Experiment
20090918 Agile Computer Control of a Complex Experiment20090918 Agile Computer Control of a Complex Experiment
20090918 Agile Computer Control of a Complex Experiment
 
Jinank Jain
Jinank JainJinank Jain
Jinank Jain
 
building intelligent systems with large scale deep learning
building intelligent systems with large scale deep learningbuilding intelligent systems with large scale deep learning
building intelligent systems with large scale deep learning
 
Exploring the Integration of User Feedback in Automated Testing of Android Ap...
Exploring the Integration of User Feedback in Automated Testing of Android Ap...Exploring the Integration of User Feedback in Automated Testing of Android Ap...
Exploring the Integration of User Feedback in Automated Testing of Android Ap...
 
Test-Driven Design Insights@DevoxxBE 2023.pptx
Test-Driven Design Insights@DevoxxBE 2023.pptxTest-Driven Design Insights@DevoxxBE 2023.pptx
Test-Driven Design Insights@DevoxxBE 2023.pptx
 
Python and Sage
Python and SagePython and Sage
Python and Sage
 
Continuous testing - GUERLAIS ARGOT - Air France KLM Sogeti- Soirée du Test L...
Continuous testing - GUERLAIS ARGOT - Air France KLM Sogeti- Soirée du Test L...Continuous testing - GUERLAIS ARGOT - Air France KLM Sogeti- Soirée du Test L...
Continuous testing - GUERLAIS ARGOT - Air France KLM Sogeti- Soirée du Test L...
 
Keynote WFIoT2019 - Data Graph, Knowledge Graphs Ontologies, Internet of Thin...
Keynote WFIoT2019 - Data Graph, Knowledge Graphs Ontologies, Internet of Thin...Keynote WFIoT2019 - Data Graph, Knowledge Graphs Ontologies, Internet of Thin...
Keynote WFIoT2019 - Data Graph, Knowledge Graphs Ontologies, Internet of Thin...
 
Automating the Quality
Automating the QualityAutomating the Quality
Automating the Quality
 
Adtech scala-performance-tuning-150323223738-conversion-gate01
Adtech scala-performance-tuning-150323223738-conversion-gate01Adtech scala-performance-tuning-150323223738-conversion-gate01
Adtech scala-performance-tuning-150323223738-conversion-gate01
 
Adtech x Scala x Performance tuning
Adtech x Scala x Performance tuningAdtech x Scala x Performance tuning
Adtech x Scala x Performance tuning
 
Jinank
JinankJinank
Jinank
 
Measuring Technical Lag in Software Deployments (CHAOSScon 2020)
Measuring Technical Lag in Software Deployments (CHAOSScon 2020)Measuring Technical Lag in Software Deployments (CHAOSScon 2020)
Measuring Technical Lag in Software Deployments (CHAOSScon 2020)
 
Quality of life through Unit Testing
Quality of life through Unit TestingQuality of life through Unit Testing
Quality of life through Unit Testing
 
Devoxx 2014 [incomplete] summary
Devoxx 2014 [incomplete] summaryDevoxx 2014 [incomplete] summary
Devoxx 2014 [incomplete] summary
 
Grokking Techtalk #38: Escape Analysis in Go compiler
 Grokking Techtalk #38: Escape Analysis in Go compiler Grokking Techtalk #38: Escape Analysis in Go compiler
Grokking Techtalk #38: Escape Analysis in Go compiler
 
Csw2016 d antoine_automatic_exploitgeneration
Csw2016 d antoine_automatic_exploitgenerationCsw2016 d antoine_automatic_exploitgeneration
Csw2016 d antoine_automatic_exploitgeneration
 

Mais de Sebastiano Panichella

Automated Identification and Qualitative Characterization of Safety Concerns ...
Automated Identification and Qualitative Characterization of Safety Concerns ...Automated Identification and Qualitative Characterization of Safety Concerns ...
Automated Identification and Qualitative Characterization of Safety Concerns ...Sebastiano Panichella
 
The 2nd Intl. Workshop on NL-based Software Engineering
The 2nd Intl. Workshop on NL-based Software EngineeringThe 2nd Intl. Workshop on NL-based Software Engineering
The 2nd Intl. Workshop on NL-based Software EngineeringSebastiano Panichella
 
The 16th Intl. Workshop on Search-Based and Fuzz Testing
The 16th Intl. Workshop on Search-Based and Fuzz TestingThe 16th Intl. Workshop on Search-Based and Fuzz Testing
The 16th Intl. Workshop on Search-Based and Fuzz TestingSebastiano Panichella
 
Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Nei...
Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Nei...Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Nei...
Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Nei...Sebastiano Panichella
 
Testing and Development Challenges for Complex Cyber-Physical Systems: Insi...
Testing and Development Challenges for  Complex Cyber-Physical Systems:  Insi...Testing and Development Challenges for  Complex Cyber-Physical Systems:  Insi...
Testing and Development Challenges for Complex Cyber-Physical Systems: Insi...Sebastiano Panichella
 
An Empirical Characterization of Software Bugs in Open-Source Cyber-Physical ...
An Empirical Characterization of Software Bugs in Open-Source Cyber-Physical ...An Empirical Characterization of Software Bugs in Open-Source Cyber-Physical ...
An Empirical Characterization of Software Bugs in Open-Source Cyber-Physical ...Sebastiano Panichella
 
COSMOS: DevOps for complex cyber-physical systems (H2020 Project) - WASOS wor...
COSMOS: DevOps for complex cyber-physical systems (H2020 Project) - WASOS wor...COSMOS: DevOps for complex cyber-physical systems (H2020 Project) - WASOS wor...
COSMOS: DevOps for complex cyber-physical systems (H2020 Project) - WASOS wor...Sebastiano Panichella
 
Exposed! A case study on the vulnerability-proneness of Google Play Apps
Exposed! A case study on the vulnerability-proneness of Google Play AppsExposed! A case study on the vulnerability-proneness of Google Play Apps
Exposed! A case study on the vulnerability-proneness of Google Play AppsSebastiano Panichella
 
Testing with Fewer Resources: Toward Adaptive Approaches for Cost-effective T...
Testing with Fewer Resources: Toward Adaptive Approaches for Cost-effective T...Testing with Fewer Resources: Toward Adaptive Approaches for Cost-effective T...
Testing with Fewer Resources: Toward Adaptive Approaches for Cost-effective T...Sebastiano Panichella
 
Search-based Software Testing (SBST) '22
Search-based Software Testing (SBST) '22Search-based Software Testing (SBST) '22
Search-based Software Testing (SBST) '22Sebastiano Panichella
 
NL-based Software Engineering (NLBSE) '22
NL-based Software Engineering (NLBSE) '22NL-based Software Engineering (NLBSE) '22
NL-based Software Engineering (NLBSE) '22Sebastiano Panichella
 
"An NLP-based Tool for Software Artifacts Analysis" at @ICSME2021.
 "An NLP-based Tool for Software Artifacts Analysis" at @ICSME2021.  "An NLP-based Tool for Software Artifacts Analysis" at @ICSME2021.
"An NLP-based Tool for Software Artifacts Analysis" at @ICSME2021. Sebastiano Panichella
 
An Empirical Investigation of Relevant Changes and Automation Needs in Modern...
An Empirical Investigation of Relevant Changes and Automation Needs in Modern...An Empirical Investigation of Relevant Changes and Automation Needs in Modern...
An Empirical Investigation of Relevant Changes and Automation Needs in Modern...Sebastiano Panichella
 
Search-Based Software Testing Tool Competition 2021 by Sebastiano Panichella,...
Search-Based Software Testing Tool Competition 2021 by Sebastiano Panichella,...Search-Based Software Testing Tool Competition 2021 by Sebastiano Panichella,...
Search-Based Software Testing Tool Competition 2021 by Sebastiano Panichella,...Sebastiano Panichella
 
A Framework for Multi-source Studies based on Unstructured Data.
A Framework for Multi-source Studies based on Unstructured Data.A Framework for Multi-source Studies based on Unstructured Data.
A Framework for Multi-source Studies based on Unstructured Data.Sebastiano Panichella
 
Revisiting Test Smells in Automatically Generated Tests: Limitations, Pitfall...
Revisiting Test Smells in Automatically Generated Tests: Limitations, Pitfall...Revisiting Test Smells in Automatically Generated Tests: Limitations, Pitfall...
Revisiting Test Smells in Automatically Generated Tests: Limitations, Pitfall...Sebastiano Panichella
 
Requirements-Collector: Automating Requirements Specification from Elicitatio...
Requirements-Collector: Automating Requirements Specification from Elicitatio...Requirements-Collector: Automating Requirements Specification from Elicitatio...
Requirements-Collector: Automating Requirements Specification from Elicitatio...Sebastiano Panichella
 
Unit Testing Tool Competition-Eighth Round
Unit Testing Tool Competition-Eighth RoundUnit Testing Tool Competition-Eighth Round
Unit Testing Tool Competition-Eighth RoundSebastiano Panichella
 

Mais de Sebastiano Panichella (20)

Automated Identification and Qualitative Characterization of Safety Concerns ...
Automated Identification and Qualitative Characterization of Safety Concerns ...Automated Identification and Qualitative Characterization of Safety Concerns ...
Automated Identification and Qualitative Characterization of Safety Concerns ...
 
The 2nd Intl. Workshop on NL-based Software Engineering
The 2nd Intl. Workshop on NL-based Software EngineeringThe 2nd Intl. Workshop on NL-based Software Engineering
The 2nd Intl. Workshop on NL-based Software Engineering
 
The 16th Intl. Workshop on Search-Based and Fuzz Testing
The 16th Intl. Workshop on Search-Based and Fuzz TestingThe 16th Intl. Workshop on Search-Based and Fuzz Testing
The 16th Intl. Workshop on Search-Based and Fuzz Testing
 
Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Nei...
Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Nei...Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Nei...
Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Nei...
 
Testing and Development Challenges for Complex Cyber-Physical Systems: Insi...
Testing and Development Challenges for  Complex Cyber-Physical Systems:  Insi...Testing and Development Challenges for  Complex Cyber-Physical Systems:  Insi...
Testing and Development Challenges for Complex Cyber-Physical Systems: Insi...
 
An Empirical Characterization of Software Bugs in Open-Source Cyber-Physical ...
An Empirical Characterization of Software Bugs in Open-Source Cyber-Physical ...An Empirical Characterization of Software Bugs in Open-Source Cyber-Physical ...
An Empirical Characterization of Software Bugs in Open-Source Cyber-Physical ...
 
COSMOS: DevOps for complex cyber-physical systems (H2020 Project) - WASOS wor...
COSMOS: DevOps for complex cyber-physical systems (H2020 Project) - WASOS wor...COSMOS: DevOps for complex cyber-physical systems (H2020 Project) - WASOS wor...
COSMOS: DevOps for complex cyber-physical systems (H2020 Project) - WASOS wor...
 
Exposed! A case study on the vulnerability-proneness of Google Play Apps
Exposed! A case study on the vulnerability-proneness of Google Play AppsExposed! A case study on the vulnerability-proneness of Google Play Apps
Exposed! A case study on the vulnerability-proneness of Google Play Apps
 
Testing with Fewer Resources: Toward Adaptive Approaches for Cost-effective T...
Testing with Fewer Resources: Toward Adaptive Approaches for Cost-effective T...Testing with Fewer Resources: Toward Adaptive Approaches for Cost-effective T...
Testing with Fewer Resources: Toward Adaptive Approaches for Cost-effective T...
 
Search-based Software Testing (SBST) '22
Search-based Software Testing (SBST) '22Search-based Software Testing (SBST) '22
Search-based Software Testing (SBST) '22
 
NL-based Software Engineering (NLBSE) '22
NL-based Software Engineering (NLBSE) '22NL-based Software Engineering (NLBSE) '22
NL-based Software Engineering (NLBSE) '22
 
NLBSE’22: Tool Competition
NLBSE’22: Tool CompetitionNLBSE’22: Tool Competition
NLBSE’22: Tool Competition
 
"An NLP-based Tool for Software Artifacts Analysis" at @ICSME2021.
 "An NLP-based Tool for Software Artifacts Analysis" at @ICSME2021.  "An NLP-based Tool for Software Artifacts Analysis" at @ICSME2021.
"An NLP-based Tool for Software Artifacts Analysis" at @ICSME2021.
 
An Empirical Investigation of Relevant Changes and Automation Needs in Modern...
An Empirical Investigation of Relevant Changes and Automation Needs in Modern...An Empirical Investigation of Relevant Changes and Automation Needs in Modern...
An Empirical Investigation of Relevant Changes and Automation Needs in Modern...
 
Search-Based Software Testing Tool Competition 2021 by Sebastiano Panichella,...
Search-Based Software Testing Tool Competition 2021 by Sebastiano Panichella,...Search-Based Software Testing Tool Competition 2021 by Sebastiano Panichella,...
Search-Based Software Testing Tool Competition 2021 by Sebastiano Panichella,...
 
A Framework for Multi-source Studies based on Unstructured Data.
A Framework for Multi-source Studies based on Unstructured Data.A Framework for Multi-source Studies based on Unstructured Data.
A Framework for Multi-source Studies based on Unstructured Data.
 
Revisiting Test Smells in Automatically Generated Tests: Limitations, Pitfall...
Revisiting Test Smells in Automatically Generated Tests: Limitations, Pitfall...Revisiting Test Smells in Automatically Generated Tests: Limitations, Pitfall...
Revisiting Test Smells in Automatically Generated Tests: Limitations, Pitfall...
 
Requirements-Collector: Automating Requirements Specification from Elicitatio...
Requirements-Collector: Automating Requirements Specification from Elicitatio...Requirements-Collector: Automating Requirements Specification from Elicitatio...
Requirements-Collector: Automating Requirements Specification from Elicitatio...
 
Unit Testing Tool Competition-Eighth Round
Unit Testing Tool Competition-Eighth RoundUnit Testing Tool Competition-Eighth Round
Unit Testing Tool Competition-Eighth Round
 
Cultural Exchange - ICSE 2020
Cultural Exchange - ICSE 2020Cultural Exchange - ICSE 2020
Cultural Exchange - ICSE 2020
 

Último

IPO OFFERINGS by mint hindustantimes.pdf
IPO OFFERINGS by mint hindustantimes.pdfIPO OFFERINGS by mint hindustantimes.pdf
IPO OFFERINGS by mint hindustantimes.pdfratnasehgal888
 
Machine learning workshop, CZU Prague 2024
Machine learning workshop, CZU Prague 2024Machine learning workshop, CZU Prague 2024
Machine learning workshop, CZU Prague 2024Gokulks007
 
Making AI Behave: Using Knowledge Domains to Produce Useful, Trustworthy Results
Making AI Behave: Using Knowledge Domains to Produce Useful, Trustworthy ResultsMaking AI Behave: Using Knowledge Domains to Produce Useful, Trustworthy Results
Making AI Behave: Using Knowledge Domains to Produce Useful, Trustworthy ResultsAccess Innovations, Inc.
 
Circle Of Life Civics Presentation Burning Issue
Circle Of Life Civics Presentation Burning IssueCircle Of Life Civics Presentation Burning Issue
Circle Of Life Civics Presentation Burning Issuebdavis22
 
Juan Pablo Sugiura - eCommerce Day Bolivia 2024
Juan Pablo Sugiura - eCommerce Day Bolivia 2024Juan Pablo Sugiura - eCommerce Day Bolivia 2024
Juan Pablo Sugiura - eCommerce Day Bolivia 2024eCommerce Institute
 
The Real Story Of Project Manager/Scrum Master From Where It Came?!
The Real Story Of Project Manager/Scrum Master From Where It Came?!The Real Story Of Project Manager/Scrum Master From Where It Came?!
The Real Story Of Project Manager/Scrum Master From Where It Came?!Loay Mohamed Ibrahim Aly
 
Communication Accommodation Theory Kaylyn Benton.pptx
Communication Accommodation Theory Kaylyn Benton.pptxCommunication Accommodation Theory Kaylyn Benton.pptx
Communication Accommodation Theory Kaylyn Benton.pptxkb31670
 
ISO 25964-1Working Group ISO/TC 46/SC 9/WG 8
ISO 25964-1Working Group ISO/TC 46/SC 9/WG 8ISO 25964-1Working Group ISO/TC 46/SC 9/WG 8
ISO 25964-1Working Group ISO/TC 46/SC 9/WG 8Access Innovations, Inc.
 
Burning Issue presentation of Zhazgul N. , Cycle 54
Burning Issue presentation of Zhazgul N. , Cycle 54Burning Issue presentation of Zhazgul N. , Cycle 54
Burning Issue presentation of Zhazgul N. , Cycle 54ZhazgulNurdinova
 
Communication Accommodation Theory Kaylyn Benton.pptx
Communication Accommodation Theory Kaylyn Benton.pptxCommunication Accommodation Theory Kaylyn Benton.pptx
Communication Accommodation Theory Kaylyn Benton.pptxkb31670
 

Último (10)

IPO OFFERINGS by mint hindustantimes.pdf
IPO OFFERINGS by mint hindustantimes.pdfIPO OFFERINGS by mint hindustantimes.pdf
IPO OFFERINGS by mint hindustantimes.pdf
 
Machine learning workshop, CZU Prague 2024
Machine learning workshop, CZU Prague 2024Machine learning workshop, CZU Prague 2024
Machine learning workshop, CZU Prague 2024
 
Making AI Behave: Using Knowledge Domains to Produce Useful, Trustworthy Results
Making AI Behave: Using Knowledge Domains to Produce Useful, Trustworthy ResultsMaking AI Behave: Using Knowledge Domains to Produce Useful, Trustworthy Results
Making AI Behave: Using Knowledge Domains to Produce Useful, Trustworthy Results
 
Circle Of Life Civics Presentation Burning Issue
Circle Of Life Civics Presentation Burning IssueCircle Of Life Civics Presentation Burning Issue
Circle Of Life Civics Presentation Burning Issue
 
Juan Pablo Sugiura - eCommerce Day Bolivia 2024
Juan Pablo Sugiura - eCommerce Day Bolivia 2024Juan Pablo Sugiura - eCommerce Day Bolivia 2024
Juan Pablo Sugiura - eCommerce Day Bolivia 2024
 
The Real Story Of Project Manager/Scrum Master From Where It Came?!
The Real Story Of Project Manager/Scrum Master From Where It Came?!The Real Story Of Project Manager/Scrum Master From Where It Came?!
The Real Story Of Project Manager/Scrum Master From Where It Came?!
 
Communication Accommodation Theory Kaylyn Benton.pptx
Communication Accommodation Theory Kaylyn Benton.pptxCommunication Accommodation Theory Kaylyn Benton.pptx
Communication Accommodation Theory Kaylyn Benton.pptx
 
ISO 25964-1Working Group ISO/TC 46/SC 9/WG 8
ISO 25964-1Working Group ISO/TC 46/SC 9/WG 8ISO 25964-1Working Group ISO/TC 46/SC 9/WG 8
ISO 25964-1Working Group ISO/TC 46/SC 9/WG 8
 
Burning Issue presentation of Zhazgul N. , Cycle 54
Burning Issue presentation of Zhazgul N. , Cycle 54Burning Issue presentation of Zhazgul N. , Cycle 54
Burning Issue presentation of Zhazgul N. , Cycle 54
 
Communication Accommodation Theory Kaylyn Benton.pptx
Communication Accommodation Theory Kaylyn Benton.pptxCommunication Accommodation Theory Kaylyn Benton.pptx
Communication Accommodation Theory Kaylyn Benton.pptx
 

How High Will It Be? Using Machine Learning Models to Predict Branch Coverage in Automated Testing

  • 1. HowHighWillItBe? Using Machine Learning Modelsto PredictBranch Coverage in AutomatedTesting G. Grano,T.Titov, S. Panichella, H. Gall MaLTeSQuE@SANER 2018, 20, Campobasso (Italy) grano@ifi.uzh.ch giograno90
  • 2. U.S. EconomyImpact1 $59.5 billion $22.2 billion avoidable 0.6% GDP 1 http://www.abeacha.com/NIST_press_release_bugs_cost.html 2 — Giovanni Grano @ s.e.a.l.
  • 3. Continuous Integration The way we develop and release code is rapidly changing In an ideal world, at every single commit the entire test suite should be executed ... ... but companies like Google commit about 16.000 changes per day! 3 — Giovanni Grano @ s.e.a.l.
  • 4. TestData Generation Pretty active research area in the last years Mature tools able to generate test suites with high coverage: > EvoSuite > Randoop > ... > and many more! How do they fit in such a CI/CD environment? 4 — Giovanni Grano @ s.e.a.l.
  • 5. Continuous Testing Generation 2 Continuous integration enhanced with automated test generation It raises many questions: > testing order > how much time to spend per class 2 Campos et al - Continuous Test Generation: Enhancing Continuous Integration with Automated Test Generation 5 — Giovanni Grano @ s.e.a.l.
  • 6. TestSuite Augmentatio n 3 Automatic generation considering code changes and their effect on the previous codebase Hardy doable to the expensive amount of time needed to generate tests 3 Xu et al - Directed Test Suite Augmentation: Techniques and Tradeoffs 6 — Giovanni Grano @ s.e.a.l.
  • 7. Coverage Prediction Knowing a priori the coverage achieved by test data generation tools > maximize the coverage for the entire system given an amount of time > budget allocation for critical components 7 — Giovanni Grano @ s.e.a.l.
  • 9. > RQ1: Which type of features can we leverage to train machine learning models to predict the branch coverage achieved by test data generation tools? > RQ2: To what extend can we predict the coverage achieved by test data generation tools? 9 — Giovanni Grano @ s.e.a.l.
  • 10. Project Selection Open Source Projects from Defect4j > Apache Cassandra > Apache Ivy > Google Guava > Google Dagger Guava Cassandr a Dagger Ivy LOC 78,525 220,573 848 50,430 Java Files 538 1,474 43 464 10 — Giovanni Grano @ s.e.a.l.
  • 11. Metrics Extractor Coverage Training Set > Run EvoSuite > Labeled data (with coverage) > Computed metrics = input variables Threats: > we did it once 11 — Giovanni Grano @ s.e.a.l.
  • 13. Package LevelFeatures Computed with JDepend 4 Name Description Ca indicator of the package's responsibility Ce indicator of the package's independence A abstract classes / total number of classes I indicator of the package's resilience to change ... ... 4 https://github.com/clarkware/jdepend 13 — Giovanni Grano @ s.e.a.l.
  • 14. CKand OO Feature ck tool provided by Aniche 5 Name Description CBO coupling between objects DIT depth of inheritance tree NOC number of children NOSF number of static field ... ... 5 https://github.com/mauricioaniche/ck 14 — Giovanni Grano @ s.e.a.l.
  • 15. Java Reserved Keywords To capture additional complexity Previously used in Information Retrieval as a feature 6 52 Java reserved keywords 6 Sanderson et al The history of information retrieval research 15 — Giovanni Grano @ s.e.a.l.
  • 16. Grid Selection best hyper-parameters 3-cross fold validation FeatureTransformation z-score 16 — Giovanni Grano @ s.e.a.l.
  • 18. Results RQ1 Huber SVR MLP Tool's Average EvoSuit e 0.255 0.216 0.242 0.238 Randoop 0.172 0.088 0.139 0.131 Average 0.213 0.152 0.191 18 — Giovanni Grano @ s.e.a.l.
  • 19. Results RQ2 Time Math Lang Tool's Aver. EvoSuit e 0.255 0.330 0.289 0.291 Randoo p 0.168 0.262 0.246 0.225 Aver. 0.211 0.296 0.267 19 — Giovanni Grano @ s.e.a.l.
  • 20. Conclusion Knowing a priori the coverage achieved by test data generation tools might ease important decisions; > we took the first steps, investigating well known features > well know features (from gut feeling) give reasonable results 20 — Giovanni Grano @ s.e.a.l.
  • 21. Futurework Improvements > larger dataset > branch-level features > feature analysis > differences between employed tools 21 — Giovanni Grano @ s.e.a.l.