Enviar pesquisa
Carregar
Mutation Testing with PIT (Booster 2014, 2014-MAR-13)
•
1 gostou
•
1,527 visualizações
Filip Van Laenen
Seguir
Tecnologia
Denunciar
Compartilhar
Denunciar
Compartilhar
1 de 62
Baixar agora
Baixar para ler offline
Recomendados
Mutation Testing
Mutation Testing
ESUG
Kill the mutants and test your tests - Roy van Rijn
Kill the mutants and test your tests - Roy van Rijn
NLJUG
Kill the mutants - A better way to test your tests
Kill the mutants - A better way to test your tests
Roy van Rijn
MUTANTS KILLER (Revised) - PIT: state of the art of mutation testing system
MUTANTS KILLER (Revised) - PIT: state of the art of mutation testing system
Tarin Gamberini
Mutation Testing
Mutation Testing
10Pines
MUTANTS KILLER - PIT: state of the art of mutation testing system
MUTANTS KILLER - PIT: state of the art of mutation testing system
Tarin Gamberini
Mutation testing in Java
Mutation testing in Java
Wojciech Langiewicz
GeeCON - Improve your tests with Mutation Testing
GeeCON - Improve your tests with Mutation Testing
Nicolas Fränkel
Recomendados
Mutation Testing
Mutation Testing
ESUG
Kill the mutants and test your tests - Roy van Rijn
Kill the mutants and test your tests - Roy van Rijn
NLJUG
Kill the mutants - A better way to test your tests
Kill the mutants - A better way to test your tests
Roy van Rijn
MUTANTS KILLER (Revised) - PIT: state of the art of mutation testing system
MUTANTS KILLER (Revised) - PIT: state of the art of mutation testing system
Tarin Gamberini
Mutation Testing
Mutation Testing
10Pines
MUTANTS KILLER - PIT: state of the art of mutation testing system
MUTANTS KILLER - PIT: state of the art of mutation testing system
Tarin Gamberini
Mutation testing in Java
Mutation testing in Java
Wojciech Langiewicz
GeeCON - Improve your tests with Mutation Testing
GeeCON - Improve your tests with Mutation Testing
Nicolas Fränkel
C++ Unit Test with Google Testing Framework
C++ Unit Test with Google Testing Framework
Humberto Marchezi
An introduction to Google test framework
An introduction to Google test framework
Abner Chih Yi Huang
Unit testing with java
Unit testing with java
Dinuka Malalanayake
Automated testing in Python and beyond
Automated testing in Python and beyond
dn
20111018 boost and gtest
20111018 boost and gtest
Will Shen
Unit Test Your Database
Unit Test Your Database
David Wheeler
3 j unit
3 j unit
kishoregali
Sample Chapter of Practical Unit Testing with TestNG and Mockito
Sample Chapter of Practical Unit Testing with TestNG and Mockito
Tomek Kaczanowski
JUnit & Mockito, first steps
JUnit & Mockito, first steps
Renato Primavera
Test driven development
Test driven development
christoforosnalmpantis
GeeCON 2012 Bad Tests, Good Tests
GeeCON 2012 Bad Tests, Good Tests
Tomek Kaczanowski
Google mock for dummies
Google mock for dummies
Harry Potter
Confitura 2012 Bad Tests, Good Tests
Confitura 2012 Bad Tests, Good Tests
Tomek Kaczanowski
Junit
Junit
FAROOK Samath
Advanced junit and mockito
Advanced junit and mockito
Mathieu Carbou
Stopping the Rot - Putting Legacy C++ Under Test
Stopping the Rot - Putting Legacy C++ Under Test
Seb Rose
Testing in-python-and-pytest-framework
Testing in-python-and-pytest-framework
Arulalan T
TDD in Python With Pytest
TDD in Python With Pytest
Eddy Reyes
أخطاء برمجية | programming errors
أخطاء برمجية | programming errors
nawal saad
Igor Popov: Mutation Testing at I T.A.K.E. Unconference 2015
Igor Popov: Mutation Testing at I T.A.K.E. Unconference 2015
Mozaic Works
Mutation testing
Mutation testing
Raúl Ávila
تحليل النظم
تحليل النظم
Kohinour Osman
Mais conteúdo relacionado
Mais procurados
C++ Unit Test with Google Testing Framework
C++ Unit Test with Google Testing Framework
Humberto Marchezi
An introduction to Google test framework
An introduction to Google test framework
Abner Chih Yi Huang
Unit testing with java
Unit testing with java
Dinuka Malalanayake
Automated testing in Python and beyond
Automated testing in Python and beyond
dn
20111018 boost and gtest
20111018 boost and gtest
Will Shen
Unit Test Your Database
Unit Test Your Database
David Wheeler
3 j unit
3 j unit
kishoregali
Sample Chapter of Practical Unit Testing with TestNG and Mockito
Sample Chapter of Practical Unit Testing with TestNG and Mockito
Tomek Kaczanowski
JUnit & Mockito, first steps
JUnit & Mockito, first steps
Renato Primavera
Test driven development
Test driven development
christoforosnalmpantis
GeeCON 2012 Bad Tests, Good Tests
GeeCON 2012 Bad Tests, Good Tests
Tomek Kaczanowski
Google mock for dummies
Google mock for dummies
Harry Potter
Confitura 2012 Bad Tests, Good Tests
Confitura 2012 Bad Tests, Good Tests
Tomek Kaczanowski
Junit
Junit
FAROOK Samath
Advanced junit and mockito
Advanced junit and mockito
Mathieu Carbou
Stopping the Rot - Putting Legacy C++ Under Test
Stopping the Rot - Putting Legacy C++ Under Test
Seb Rose
Testing in-python-and-pytest-framework
Testing in-python-and-pytest-framework
Arulalan T
TDD in Python With Pytest
TDD in Python With Pytest
Eddy Reyes
Mais procurados
(18)
C++ Unit Test with Google Testing Framework
C++ Unit Test with Google Testing Framework
An introduction to Google test framework
An introduction to Google test framework
Unit testing with java
Unit testing with java
Automated testing in Python and beyond
Automated testing in Python and beyond
20111018 boost and gtest
20111018 boost and gtest
Unit Test Your Database
Unit Test Your Database
3 j unit
3 j unit
Sample Chapter of Practical Unit Testing with TestNG and Mockito
Sample Chapter of Practical Unit Testing with TestNG and Mockito
JUnit & Mockito, first steps
JUnit & Mockito, first steps
Test driven development
Test driven development
GeeCON 2012 Bad Tests, Good Tests
GeeCON 2012 Bad Tests, Good Tests
Google mock for dummies
Google mock for dummies
Confitura 2012 Bad Tests, Good Tests
Confitura 2012 Bad Tests, Good Tests
Junit
Junit
Advanced junit and mockito
Advanced junit and mockito
Stopping the Rot - Putting Legacy C++ Under Test
Stopping the Rot - Putting Legacy C++ Under Test
Testing in-python-and-pytest-framework
Testing in-python-and-pytest-framework
TDD in Python With Pytest
TDD in Python With Pytest
Destaque
أخطاء برمجية | programming errors
أخطاء برمجية | programming errors
nawal saad
Igor Popov: Mutation Testing at I T.A.K.E. Unconference 2015
Igor Popov: Mutation Testing at I T.A.K.E. Unconference 2015
Mozaic Works
Mutation testing
Mutation testing
Raúl Ávila
تحليل النظم
تحليل النظم
Kohinour Osman
Re charge the interview
Re charge the interview
Claudio De La Parra
Doctor of Philosophy
Doctor of Philosophy
rashidw
Costituciones y pacto social
Costituciones y pacto social
Giovany2015
Guide aides financieres renovation habitat - juillet -2016
Guide aides financieres renovation habitat - juillet -2016
Isocell France
שנה טובה מישראל נגלית לעין
שנה טובה מישראל נגלית לעין
israelalbum_ybz
Introduction to the Global Crop Diversity Trust
Introduction to the Global Crop Diversity Trust
Luigi Guarino
5 d nl11 unidad2 3
5 d nl11 unidad2 3
Lab2computacion2
Preguntas 3-encu-clau
Preguntas 3-encu-clau
Diego Solano
Brian Ramirez-Letter of Recommendation
Brian Ramirez-Letter of Recommendation
Brian Ramirez
АТОЛ. Презентация 1
АТОЛ. Презентация 1
MoySklad
Gramática - Advérbio
Gramática - Advérbio
Mentor Concursos
The Human Brain
The Human Brain
annhmoore
Simon White Showreel
Simon White Showreel
Simon White
2015 05 heimdall-joint-brochure-v03
2015 05 heimdall-joint-brochure-v03
Mark Repton
Maledivy 2
Maledivy 2
PetkaHuskova
Destaque
(19)
أخطاء برمجية | programming errors
أخطاء برمجية | programming errors
Igor Popov: Mutation Testing at I T.A.K.E. Unconference 2015
Igor Popov: Mutation Testing at I T.A.K.E. Unconference 2015
Mutation testing
Mutation testing
تحليل النظم
تحليل النظم
Re charge the interview
Re charge the interview
Doctor of Philosophy
Doctor of Philosophy
Costituciones y pacto social
Costituciones y pacto social
Guide aides financieres renovation habitat - juillet -2016
Guide aides financieres renovation habitat - juillet -2016
שנה טובה מישראל נגלית לעין
שנה טובה מישראל נגלית לעין
Introduction to the Global Crop Diversity Trust
Introduction to the Global Crop Diversity Trust
5 d nl11 unidad2 3
5 d nl11 unidad2 3
Preguntas 3-encu-clau
Preguntas 3-encu-clau
Brian Ramirez-Letter of Recommendation
Brian Ramirez-Letter of Recommendation
АТОЛ. Презентация 1
АТОЛ. Презентация 1
Gramática - Advérbio
Gramática - Advérbio
The Human Brain
The Human Brain
Simon White Showreel
Simon White Showreel
2015 05 heimdall-joint-brochure-v03
2015 05 heimdall-joint-brochure-v03
Maledivy 2
Maledivy 2
Semelhante a Mutation Testing with PIT (Booster 2014, 2014-MAR-13)
Oop 2015 – Mutation Testing
Oop 2015 – Mutation Testing
Filip Van Laenen
How good are your tests?
How good are your tests?
Noam Shaish
Mike Bartley - Innovations for Testing Parallel Software - EuroSTAR 2012
Mike Bartley - Innovations for Testing Parallel Software - EuroSTAR 2012
TEST Huddle
SSBSE 2020 keynote
SSBSE 2020 keynote
Shiva Nejati
Shift-Left Testing: QA in a DevOps World by David Laulusa
Shift-Left Testing: QA in a DevOps World by David Laulusa
QA or the Highway
Automatic Test Pattern Generation (Testing of VLSI Design)
Automatic Test Pattern Generation (Testing of VLSI Design)
Usha Mehta
Unit Testng with PHP Unit - A Step by Step Training
Unit Testng with PHP Unit - A Step by Step Training
Ram Awadh Prasad, PMP
Performance Issue? Machine Learning to the rescue!
Performance Issue? Machine Learning to the rescue!
Maarten Smeets
Model-Based Testing: Theory and Practice. Keynote @ MoTiP (ISSRE) 2012.
Model-Based Testing: Theory and Practice. Keynote @ MoTiP (ISSRE) 2012.
Wolfgang Grieskamp
Seven deadly sins of ElasticSearch Benchmarking
Seven deadly sins of ElasticSearch Benchmarking
Fan Robbin
Distributed Model Validation with Epsilon
Distributed Model Validation with Epsilon
Sina Madani
Avoiding test hell
Avoiding test hell
Yun Ki Lee
Performance Testing Java Applications
Performance Testing Java Applications
C4Media
Testing of Cyber-Physical Systems: Diversity-driven Strategies
Testing of Cyber-Physical Systems: Diversity-driven Strategies
Lionel Briand
Chapter 14 software testing techniques
Chapter 14 software testing techniques
SHREEHARI WADAWADAGI
TMPA-2017: Live testing distributed system fault tolerance with fault injecti...
TMPA-2017: Live testing distributed system fault tolerance with fault injecti...
Iosif Itkin
NIO-ICSE2022.pptx
NIO-ICSE2022.pptx
DavidWei89
It Does What You Say, Not What You Mean: Lessons From A Decade of Program Repair
It Does What You Say, Not What You Mean: Lessons From A Decade of Program Repair
Claire Le Goues
Mutation Testing.pdf
Mutation Testing.pdf
Keir Bowden
Unit testing basics
Unit testing basics
João Paulo Leonidas Fernandes Dias da Silva
Semelhante a Mutation Testing with PIT (Booster 2014, 2014-MAR-13)
(20)
Oop 2015 – Mutation Testing
Oop 2015 – Mutation Testing
How good are your tests?
How good are your tests?
Mike Bartley - Innovations for Testing Parallel Software - EuroSTAR 2012
Mike Bartley - Innovations for Testing Parallel Software - EuroSTAR 2012
SSBSE 2020 keynote
SSBSE 2020 keynote
Shift-Left Testing: QA in a DevOps World by David Laulusa
Shift-Left Testing: QA in a DevOps World by David Laulusa
Automatic Test Pattern Generation (Testing of VLSI Design)
Automatic Test Pattern Generation (Testing of VLSI Design)
Unit Testng with PHP Unit - A Step by Step Training
Unit Testng with PHP Unit - A Step by Step Training
Performance Issue? Machine Learning to the rescue!
Performance Issue? Machine Learning to the rescue!
Model-Based Testing: Theory and Practice. Keynote @ MoTiP (ISSRE) 2012.
Model-Based Testing: Theory and Practice. Keynote @ MoTiP (ISSRE) 2012.
Seven deadly sins of ElasticSearch Benchmarking
Seven deadly sins of ElasticSearch Benchmarking
Distributed Model Validation with Epsilon
Distributed Model Validation with Epsilon
Avoiding test hell
Avoiding test hell
Performance Testing Java Applications
Performance Testing Java Applications
Testing of Cyber-Physical Systems: Diversity-driven Strategies
Testing of Cyber-Physical Systems: Diversity-driven Strategies
Chapter 14 software testing techniques
Chapter 14 software testing techniques
TMPA-2017: Live testing distributed system fault tolerance with fault injecti...
TMPA-2017: Live testing distributed system fault tolerance with fault injecti...
NIO-ICSE2022.pptx
NIO-ICSE2022.pptx
It Does What You Say, Not What You Mean: Lessons From A Decade of Program Repair
It Does What You Say, Not What You Mean: Lessons From A Decade of Program Repair
Mutation Testing.pdf
Mutation Testing.pdf
Unit testing basics
Unit testing basics
Mais de Filip Van Laenen
Drawing for IT Architects
Drawing for IT Architects
Filip Van Laenen
How JSR 385 could have saved the Mars Climate Orbiter
How JSR 385 could have saved the Mars Climate Orbiter
Filip Van Laenen
How JSR-385 Could Have Saved the Mars Climate Orbiter
How JSR-385 Could Have Saved the Mars Climate Orbiter
Filip Van Laenen
Clouds with Trenches and Sharp Edges
Clouds with Trenches and Sharp Edges
Filip Van Laenen
Become an SVG Architect, not a PowerPoint Architect
Become an SVG Architect, not a PowerPoint Architect
Filip Van Laenen
Dial M for Mutation
Dial M for Mutation
Filip Van Laenen
Mutasjonstesting – Lag bugs for å få bedre kode
Mutasjonstesting – Lag bugs for å få bedre kode
Filip Van Laenen
Hvem kommer til å vinne kommunevalget?
Hvem kommer til å vinne kommunevalget?
Filip Van Laenen
Five Inconvenient Truths about REST
Five Inconvenient Truths about REST
Filip Van Laenen
How Free Data Can Drive Some of the Monkey Business Out of Political Journali...
How Free Data Can Drive Some of the Monkey Business Out of Political Journali...
Filip Van Laenen
#NoEstimates – Smidig 2014
#NoEstimates – Smidig 2014
Filip Van Laenen
#NoEstimates – Javazone 2014
#NoEstimates – Javazone 2014
Filip Van Laenen
Tre ubeleilige sannheter om REST
Tre ubeleilige sannheter om REST
Filip Van Laenen
What Architects Really Do
What Architects Really Do
Filip Van Laenen
Hvorfor stole på e-valg 2011/13?
Hvorfor stole på e-valg 2011/13?
Filip Van Laenen
Mutation testing (OOP 2012, 2012-JAN-24)
Mutation testing (OOP 2012, 2012-JAN-24)
Filip Van Laenen
SVG (Devoxx 2011, 2011-NOV-14)
SVG (Devoxx 2011, 2011-NOV-14)
Filip Van Laenen
SVG (Framsia, 27-SEP-2011)
SVG (Framsia, 27-SEP-2011)
Filip Van Laenen
Mais de Filip Van Laenen
(18)
Drawing for IT Architects
Drawing for IT Architects
How JSR 385 could have saved the Mars Climate Orbiter
How JSR 385 could have saved the Mars Climate Orbiter
How JSR-385 Could Have Saved the Mars Climate Orbiter
How JSR-385 Could Have Saved the Mars Climate Orbiter
Clouds with Trenches and Sharp Edges
Clouds with Trenches and Sharp Edges
Become an SVG Architect, not a PowerPoint Architect
Become an SVG Architect, not a PowerPoint Architect
Dial M for Mutation
Dial M for Mutation
Mutasjonstesting – Lag bugs for å få bedre kode
Mutasjonstesting – Lag bugs for å få bedre kode
Hvem kommer til å vinne kommunevalget?
Hvem kommer til å vinne kommunevalget?
Five Inconvenient Truths about REST
Five Inconvenient Truths about REST
How Free Data Can Drive Some of the Monkey Business Out of Political Journali...
How Free Data Can Drive Some of the Monkey Business Out of Political Journali...
#NoEstimates – Smidig 2014
#NoEstimates – Smidig 2014
#NoEstimates – Javazone 2014
#NoEstimates – Javazone 2014
Tre ubeleilige sannheter om REST
Tre ubeleilige sannheter om REST
What Architects Really Do
What Architects Really Do
Hvorfor stole på e-valg 2011/13?
Hvorfor stole på e-valg 2011/13?
Mutation testing (OOP 2012, 2012-JAN-24)
Mutation testing (OOP 2012, 2012-JAN-24)
SVG (Devoxx 2011, 2011-NOV-14)
SVG (Devoxx 2011, 2011-NOV-14)
SVG (Framsia, 27-SEP-2011)
SVG (Framsia, 27-SEP-2011)
Último
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other Frameworks
Softradix Technologies
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
HampshireHUG
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101
Paola De la Torre
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
Pixlogix Infotech
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
hans926745
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
Delhi Call girls
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
Delhi Call girls
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
HostedbyConfluent
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
Gabriella Davis
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
Delhi Call girls
Key Features Of Token Development (1).pptx
Key Features Of Token Development (1).pptx
LBM Solutions
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
Neo4j
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
2toLead Limited
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
Puma Security, LLC
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
ThousandEyes
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptx
OnBoard
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
Malak Abu Hammad
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
naman860154
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
soniya singh
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
Ridwan Fadjar
Último
(20)
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other Frameworks
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
Key Features Of Token Development (1).pptx
Key Features Of Token Development (1).pptx
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
Mutation Testing with PIT (Booster 2014, 2014-MAR-13)
1.
© Computas AS
11.03.14 Mutation Testing with PIT Filip van Laenen Booster 2014 2014-03-13
2.
2 © Computas
AS 11.03.14 Agenda • Basics of mutation testing • Relation to other testing techniques • Example • Mutation testing techniques • Mutation testing tools • Personal experiences and recommendations
3.
3 © Computas
AS 11.03.14 Basics of Mutation Testing
4.
4 © Computas
AS 11.03.14 Mutation Testing in a Nutshell Seeking The Summoner @ The Daily WTF http://thedailywtf.com/Articles/Seeking-The-Summoner.aspx
5.
5 © Computas
AS 11.03.14 Mutation Testing in a Nutshell (cont'd)
6.
6 © Computas
AS 11.03.14 Mutation Testing in a Nutshell (cont'd)
7.
7 © Computas
AS 11.03.14 Mutation Testing in a Nutshell (cont'd) • Unit tests guard the source code • But who guards the guardians? • Do the unit tests cover all source code? • Lines? • Branches? • Paths? • Do the unit tests test the right things? Mutation testing tests the tests!
8.
8 © Computas
AS 11.03.14 Mutation Testing in a Nutshell (cont'd)
9.
9 © Computas
AS 11.03.14 Mutation Testing in a Nutshell (cont'd)
10.
10 © Computas
AS 11.03.14 Mutation Testing in a Nutshell (cont'd)
11.
11 © Computas
AS 11.03.14 Mutation Testing in a Nutshell (cont'd)
12.
12 © Computas
AS 11.03.14 Mutation Testing in a Nutshell (cont'd)
13.
13 © Computas
AS 11.03.14 Mutation Testing in a Nutshell (cont'd)
14.
14 © Computas
AS 11.03.14 Mutation Testing in a Nutshell (cont'd) int max(int a, int b) { return (a < b) ? b : a; } int max(int a, int b) { return (a <= b) ? b : a; }
15.
15 © Computas
AS 11.03.14 Mutation Testing in a Nutshell (cont'd)
16.
“ 16 © Computas
AS 11.03.14 Does Mutation Testing Work? In practice, if the software contains a fault, there will usually be a set of mutants that can only be killed by a test case that also detects that fault. Geist et. al., “Estimation and Enhancement of Real-time Software Reliability through Mutation Analysis,” 1992
17.
“ 17 © Computas
AS 11.03.14 Does Mutation Testing Work? (cont'd) Complex faults are coupled to simple faults in such a way that a test data set that detects all simple faults in a program will detect most complex faults. K. Wah, “Fault Coupling in Finite Bijective Functions,” 1995
18.
18 © Computas
AS 11.03.14 Does Mutation Testing Work? (cont'd) • “Generated mutants are similar to real faults.” • Andrews, Briand, Labiche, ICSE 2005 • “Mutation testing is more powerful than statement or branch coverage.” • Walsh, Ph.D. Thesis, State University of New York at Binghampton, 1985 • “Mutation testing is superior to data flow coverage criteria.” • Frankl, Weiss, Hu, Journal of Systems and Software, 1997
19.
19 © Computas
AS 11.03.14 Relation to Other Testing Techniques
20.
20 © Computas
AS 11.03.14 Relation to Other Testing Techniques • Unit tests • Test-Driven Development (TDD) • Test coverage • Static code analysis • Fuzz testing
21.
21 © Computas
AS 11.03.14 Relation to Other Testing Techniques
22.
22 © Computas
AS 11.03.14 Is Mutation Testing New? • R. Lipton, “Fault Diagnosis of Computer Programs,” 1971 • R. Lipton et. al., “Hints on Test Data Selection: Help for the Practicing Programmer,” 1978 • Historical obstacles: • No unit testing • No TDD • Inefficient implementation • Hence time-consuming • No integration with IDEs
23.
23 © Computas
AS 11.03.14 Practical Example of Mutation Testing
24.
24 © Computas
AS 11.03.14 Code Along… https://github.com/computas-fvl/booster2014 git clone https://github.com/computas-fvl/booster2014.git mvn clean site:site org.pitest:pitest-maven:mutationCoverage
25.
25 © Computas
AS 11.03.14 Mutation Testing Techniques
26.
26 © Computas
AS 11.03.14 Mutation Testing Techniques • Three aspects: • Mutation injection • Mutation types • Unit test selection per mutant • Key properties: • Efficiency • Performance
27.
27 © Computas
AS 11.03.14 Mutation Injection • Source code mutation • Binary code mutation • Caveats: • De-mutation • Compilation errors • Invalid binary code
28.
28 © Computas
AS 11.03.14 Mutation Types • Some mutations never change behaviour • Constants reused by unit tests • Log messages • Some mutations can change behaviour • Switching between < and ≠ • Switching between < and ≤ • Some mutations always change behaviour • Switching between < and ≥
29.
29 © Computas
AS 11.03.14 Mutation Types (cont'd) int max(int a, int b) { return (a < b) ? b : a; } int max(int a, int b) { return (a <= b) ? b : a; } int max(int a, int b) { return (a >= b) ? b : a; }
30.
30 © Computas
AS 11.03.14 Mutation Types (cont'd) for (int i = 0; i < 10; i++) … for (int i = 0; i != 10; i++) … for (int i = 0; i >= 10; i++) …
31.
31 © Computas
AS 11.03.14 Mutation Types Guaranteed to Change Behaviour • Negation of the comparison • Switching between = and ≠ • Switching between < and ≥ • Switching between > and ≤ • Negation of boolean conditions • Adding a ¬, ! or ~ • Shortcutting boolean conditions • Replacement with True or False *
32.
32 © Computas
AS 11.03.14 Unit Test Selection • Goal: find the unit test that “kills” the mutant • Selection aids: • Hints • Name/package matching • Code coverage tools • Automatic learning • Other heuristics
33.
33 © Computas
AS 11.03.14 Unit Test Selection (cont'd) • System: • 50 classes • 20 unit tests per class • 1 ms per unit test • Unit testing time: 50 × 20 × 1ms = 1s • 10 mutants per class: • Brute-force: 10 × 50 × 1s = 6m 20s • Educated: 10 × 50 × 20 × 1ms = 10s
34.
34 © Computas
AS 11.03.14 Unit Test Selection (cont'd) • System: • 500 classes • 20 unit tests per class • 1 ms per unit test • Unit testing time: 500 × 20 × 1ms = 10s • 10 mutants per class: • Brute-force: 10 × 500 × 10s = 13h 53m 20s • Educated: 10 × 500 × 20 × 1ms = 1m 40s
35.
35 © Computas
AS 11.03.14 Complexity • f: Number of function points • φ: Number of function points per class, ≥ 1 • τ: Number of unit tests per function point, ≥ 1 • μ: Number of mutants per function point, ≥ 1 • Brute-force: (f × τ) × (f × μ) = τ × μ × f² • Educated force: τ × μ × φ × f • Ideal: τ × μ × f
36.
36 © Computas
AS 11.03.14 Complexity (cont'd) • c: Number of classes • φ: Number of function points per class, ≥ 1 • τ: Number of unit tests per function point, ≥ 1 • μ: Number of mutants per function point, ≥ 1 • Brute-force: (f × τ) × (f × μ) = τ × μ × φ² × c² • Educated force: τ × μ × φ² × c • Ideal: τ × μ × φ × c
37.
37 © Computas
AS 11.03.14 Loops for (int i = 0; i < 10; i++) … for (int i = 0; i < 10; i--) …
38.
38 © Computas
AS 11.03.14 Infinite Loops • Terminate mutants that take too long to run • What's “too long”? • Ruins the performance • Can be hard to predict
39.
39 © Computas
AS 11.03.14 Other Problems • Recursion: • Stack overflows • Out of memory exceptions • Syntax errors • Segmentation faults
40.
40 © Computas
AS 11.03.14 Mutation Testing Tools
41.
41 © Computas
AS 11.03.14 Mutation Testing Tools • Java: • PIT • Jester (Nester, Pester) • Jumble • Ruby: • Mutant • Heckle • C#: • NinjaTurtles
42.
42 © Computas
AS 11.03.14 PIT • Java • Junit & TestNG • Maven or command-line • Operates on byte code • Large set of mutators • Also possibly equivalent mutators available • Highly configurable • Sensible defaults • http://pitest.org/
43.
43 © Computas
AS 11.03.14 PIT Mutators • Conditionals Boundary • Negate Conditionals • Remove Conditionals* • Math • Increments • Invert Negatives • Inline Constant* • Return Values • Void Method Calls • Non Void Method Calls* • Constructor Calls*
44.
44 © Computas
AS 11.03.14 PIT Sample Report
45.
45 © Computas
AS 11.03.14 Jester • Java • JUnit • Usually run from the command-line • Grester for Maven2 • Operates on source code • Runs all unit tests on all classes • Reporting and documentation could be better • http://jester.sourceforge.net/ • http://sourceforge.net/projects/grester/
46.
46 © Computas
AS 11.03.14 Jester Sample Report Overview
47.
47 © Computas
AS 11.03.14 Jester Sample Detailed Report
48.
48 © Computas
AS 11.03.14 Pester and Nester • Pester • Jester for Python • PyUnit • Nester • Port of Jester for C# • NUnit • Integrated with Visual Studio • But outdated… • http://nester.sourceforge.net/
49.
49 © Computas
AS 11.03.14 Nester Sample Report
50.
50 © Computas
AS 11.03.14 Jumble • Java • JUnit • Run from the command-line • Operates on byte code • Runs unit tests on a class • Reporting and documentation could be better • Claims to be faster than Jester • http://jumble.sourceforge.net/index.html
51.
51 © Computas
AS 11.03.14 Jumble Sample Report Mutating Foo Tests: FooTest Mutation points = 12, unit test time limit 2.02s .. M FAIL: Foo:31: negated conditional M FAIL: Foo:33: negated conditional M FAIL: Foo:34: - -> + M FAIL: Foo:35: negated conditional ...... Score: 67%
52.
52 © Computas
AS 11.03.14 Mutant • Ruby 1.9 and 2.0 • rspec2 • Usually run from the command-line • Good to-the-point reporting • Good performance • OK documentation • Active project • https://github.com/mbj/mutant
53.
53 © Computas
AS 11.03.14 Heckle • Ruby 1.8 • Doesn't work on Ruby 1.9 • Test::Unit and rSpec • Usually run from the command-line • Runs a set of unit tests on a class or a method • Good to-the-point reporting • Good performance • Virtually no documentation • http://rubyforge.org/projects/seattlerb/ • http://docs.seattlerb.org/heckle/
54.
54 © Computas
AS 11.03.14 Heckle Mutations • Booleans • Numbers • Strings • Symbols • Ranges • Regexes • Branches (if, while, unless, until)
55.
55 © Computas
AS 11.03.14 NinjaTurtles • .Net: • C# • Visual Basic/VB.NET • Any other .NET language code • Doesn't seem much alive • http://www.mutation-testing.net/
56.
56 © Computas
AS 11.03.14 NinjaTurtles Mutations • Sequence point deletion • Arithmetic operator substitution (*, /, +, -, %) • Bitwise operator substitution (&, |, ^) • Branch substituion (condition, always and never branch) • Conditional boundary substition (< and <=, > and >=) • Substitution of reads from variables, parameters and fields of the same type • Substitution of writes to variables of the same type
57.
57 © Computas
AS 11.03.14 Personal Experiences and Recommendations
58.
58 © Computas
AS 11.03.14 Experiences and Recommendations • Use mutation testing from day 1 • Start on a small code base • Keep number of unit tests per class low • Have small classes • Select a good tool • Configurable • Flexible • One that can output the mutant
59.
59 © Computas
AS 11.03.14 Experiences and Recommendations (cont'd) • Believe the tool • Or try to proof that the tool is wrong • Fix the problem • Don't turn mutation testing off • Embrace the “more than 100%” test coverage • Path coverage • Less code • More unit tests • More intelligent unit tests
60.
60 © Computas
AS 11.03.14 Improvements
61.
61 © Computas
AS 11.03.14 Improvements • Integration with more unit testing frameworks • Better unit test–source code mapping • Better heuristics • Parallellisation • Better reporting • IDE integration • Building tool integration
62.
62 © Computas
AS 11.03.14 Questions? Computas AS Lysaker Torg 45, pb 482 N-1327 Lysaker NORWAY Tel +47-67 83 10 00 Fax +47-67 83 10 01 Org.nr: NO 986 352 325 MVA www.computas.com Contact: fvl@computas.com @filipvanlaenen
Baixar agora