Panel introduction slides

1. Panel: Past and Future of Software
Testing and Analysis
Lionel Briand
http://www.lbriand.info
ISSTA 2021

2. Contributions in the last decade
• Search-based software testing
• E.g., scalable test data generation, e.g., at system level
• Use of Machine learning in test automation
• E.g., Regression test selection and prioritization
• Use of test execution history and change data across builds
• In continuous development and integration environments
• Testing cyber-physical systems
• Testing CPS models (falsification), simulation-driven testing
• Trace analysis and monitoring, e.g., specification languages
2

3. Future directions
• We have developed many interesting ideas and concepts
• Many of them are not practical or scalable, in most
contexts
• For example: mutation testing, metamorphic testing, etc.
• We need to focus on devising novel engineering solutions
that are scalable and practical, at least in some well-
defined contexts
• E.g., mutation analysis in embedded systems in the space
domain
3

4. Future directions
• Multi-disciplinary research: It is rarely the case that one
technology solves an actual problem
• We have many performant technologies at our disposal,
that have made big leaps forward in the last decade:
• Machine learning
• Natural Language Processing
• Meta-heuristic search
• Solvers (e.g., SMT)
• Symbolic execution
• Simulation
• How to combine them in a specific context for targeted
problems? This is hard work, e.g., Automotive DS.
4

5. Critical factors for the field
• We need fundamental research, not driven by specific
problems, and we have done that well as a community
• However, our practical impact has been limited and,
over time, this is being noticed by institutions, funding
agencies, etc.
• Other fields have had much more (visible) impact
• Testing and analysis research is a practical field that
should have substantial visible impact
5

6. Critical factors for the field
• Impact requires a thorough understanding of problems
• Present and foreseeable problems
• Problem understanding requires collaboration between
researchers and industry
• Collaboration needs to be supported and encouraged
by our community, academic institutions, and funding
agencies
6

7. Applied research, driven by
industrial contexts, focused
on scalable and applicable
solutions, is difficult and
rewarding research work
7

8. Panel: Past and Future of Software
Testing and Analysis
Lionel Briand
http://www.lbriand.info
ISSTA 2021