Automated Identification and Qualitative Characterization of Safety Concerns Reported in UAV Software Platforms

Sebastiano Panichella
Sebastiano PanichellaSenior Computer Science Researcher em Zurich University of Applied Science
J1 presentation:
“Automated Identification and Qualitative Characterization of Safety Concerns
Reported in UAV Software Platforms”
Transactions on Software Engineering and Methodology.
Mon 11 - Fri 15 September 2023, Luxembourg
Sebastiano Panichella
Zurich University of Applied Sciences
https://spanichella.github.io/
Andrea Di Sorbo Fiorella Zampetti Aaron Visaggio Massimiliano Di Penta
University of Sannio
IEEE/ACM International Conference on
Automated Software Engineering (ASE 2023)
Outline
2
• DevOps shortcomings for Complex CPSs
• What types of safety bugs occur in open-source CPSs?
• Context: Cyber-physical Systems (CPSs)
The COSMOS Project has
received funding from
the European Union’s
Horizon 2020 Research
and Innovation
Programme under grant
agreement No. 957254.
UAV
“Emerging Cyber-physical Systems (CPS) will play a crucial role in the quality of
life of European citizens and the future of the European economy”
Context
• CPS relevant sectors:
• Healthcare
• Automotive
• Water Monitoring
• Railway
• Manufacturing
• Avionics
• etc.
MEDICAL DELIVERY
FOOD DELIVERY
• Avionics
3
FOOD DELIVERY
Background
First aerodynamic flight on another planet. Landed with Perseverance rover on 18 February 2021
SPACE EXPLORATION
• -
• Our (Software Engineering) view of DevOps and AI for IoT systems:
• DevOps and Continuous Delivery (CD): Whats is it?
• Present, Challenges, and Opportunities
• Relevant Research Questions
• Artificial Intelligence (AI) and Testing Automation:
• Present, Challenges, and Opportunities
• User-oriented Testing Automation
• Relevant Research Questions
“We all recognize the relevance and capacity of contemporary cyber-
physical systems for building the future of our society, but ongoing research
in the field is also clearly failing in making the right countermeasures to
avoid that CPS usage affects human being safety”. In
“Self-driving Uber kills Arizona
woman in first fatal crash involving
pedestrian”
“Swiss Post drone
crashes in Zurich
Problem Statement
“A simple software update was
the direct cause of the fatal
crashes of the Boeing 737”
6
• -
• Our (Software Engineering) view of DevOps and AI for IoT systems:
• DevOps and Continuous Delivery (CD): Whats is it?
• Present, Challenges, and Opportunities
• Relevant Research Questions
• Artificial Intelligence (AI) and Testing Automation:
• Present, Challenges, and Opportunities
• User-oriented Testing Automation
• Relevant Research Questions
“Self-driving Uber kills Arizona
woman in first fatal crash involving
pedestrian”
“Swiss Post drone
crashes in Zurich
Challenge
“A simple software update was
the direct cause of the fatal
crashes of the Boeing 737”
Observability, testability, and predictability of the behavior of emerging
CPS is highly limited and, unfortunately, their usage in the real world can
lead to fatal crashes sometimes tragically involving also humans
7
Sebastiano Panichella Sajad Khatiri
Christian Birchler
COSMOS:
DevOps for Complex Cyber-physical Systems
https://www.cosmos-devops.org/ https://twitter.com/COSMOS_DEVOPS https://lnkd.in/eUVeaYaz
9
What types of bugs occur in open-source CPSs?
CPS
Bugs
10
Famous Software Failure in a CPS: The Case of Ariane 5
11
Bugs and Failures in CPS
CPS Bug is
“a flaw in the hardware (not properly
handled by the software), or an incorrect
interaction between the software and
hardware components leading to a CPS
misbehavior’’
A CPS bug can manifest as a CPS failure,
which makes a CPS unable to deliver its
required functionality or not fulfilling some
non-functional properties
Properties
12
Bugs in the PX4 Project
https://github.com/PX4/PX4-Autopilot/issues/8980
Px4 Issue 8980: Unsuccessful flight
“ Autopilot receiving noisy sensor-data…“
13
Fiorella Zampetti, Ritu Kapur, Massimiliano Di Penta,
Sebastiano Panichella: An Empirical Characterization
of Software Bugs in Open-Source Cyber-Physical
Systems. Journal of Systems & Software (JSS).
What types of bugs occur in Open-source CPSs?
CPS
Bugs
1,151 closed issues sampled from
14 open-source CPS projects
CPS bugs taxonomy comprises
8 different high-level categories
17
What about Safety Bugs of CPSs?
CPS
Bugs
18
What are the main Hazards and Accidents Emerging from Safety Issues
Reported in UAV Software Platforms?
Andrea Di Sorbo, Fiorella Zampetti, Corrado A. Visaggio, Massimiliano Di Penta, and Sebastiano
Panichella: Automated Identification and Qualitative Characterization of Safety Concerns Reported in
UAV Software Platforms. Transactions on Software Engineering and Methodology.
150 Projects
19
What are the main Hazards and Accidents Emerging from Safety Issues
Reported in UAV Software Platforms?
CPS Safety Related Issues of UAVs
Andrea Di Sorbo, Fiorella Zampetti, Corrado A. Visaggio, Massimiliano Di Penta, and Sebastiano
Panichella: Automated Identification and Qualitative Characterization of Safety Concerns Reported in
UAV Software Platforms. Transactions on Software Engineering and Methodology.
20
RQ1: To what extent can machine learning models
automatically identify safety-related concerns in
issue reports of UAV software platforms?
RQ2: What are the main hazards and accidents emerging
from safety issues reported in UAV software platforms?
CPS Safety Related Issues of UAVs
RQ1: To what extent can machine learning models
automatically identify safety-related concerns in
issue reports of UAV software platforms?
RQ2: What are the main hazards and accidents emerging
from safety issues reported in UAV software platforms?
What are the main Hazards and Accidents Emerging from Safety Issues Reported in UAV
Software Platforms?
21
Gold Standard Definition
1,916 sentences
Cohen’s k = 0.68
(substantial)
837 (43.7%) safety
1,079 (56.3%) non-safety
22
Safety Bugs in open-source CPSs
Preprocessing of CPS Safety Related Issues
Tf Matrix
GitHub
Issues
Tf Matrix Tf-idf Matrix
RQ1: To what extent can machine learning models
automatically identify safety-related concerns in
issue reports of UAV software platforms?
RQ2: What are the main hazards and accidents emerging
from safety issues reported in UAV software platforms?
23
Safety Bugs in open-source CPSs
ML prediction & Hyperparameter optimization
Tf Matrix
Tf-idf Matrix
RQ1: To what extent can machine learning models
automatically identify safety-related concerns in
issue reports of UAV software platforms?
RQ2: What are the main hazards and accidents emerging
from safety issues reported in UAV software platforms?
24
Safety Bugs in open-source CPSs
Performance of the ML classifiers
for classifying safety-related
issues in pull requests
Tf-idf wins over other
processing approaches
> 80% Precision,
Recall, and F-measure (10-
fold validation) using word
embeddings with FastText
Link to the tool behind our research
https://github.com/spanichella/SALIENT-TOOL
26
What are the main Hazards and Accidents Emerging from Safety Issues
Reported in UAV Software Platforms?
273 safety-related issues and PR 19 Hazard categories
7 Accident categories
27
Co-occurrences
of hazard
categories
and accident
categories
Hazard Accident
Hazard categories and
corresponding occurrences in our
dataset of 273 safety-related
issues and pull requests.
What are the main Hazards and Accidents Emerging from Safety Issues
Reported in UAV Software Platforms?
What are the main Hazards and Accidents Emerging from Safety Issues
Reported in UAV Software Platforms?
29
Co-occurrences
of hazard
categories
and accident
categories
Hazard Accident
Hazard categories and
corresponding occurrences in our
dataset of 273 safety-related
issues and pull requests.
Main Findings:
We need automated approaches to monitor and address safety concerns within a
CPS DevOps pipeline
What are the main Hazards and Accidents Emerging from Safety Issues
Reported in UAV Software Platforms?
Summary
30
• DevOps shortcomings for Complex CPSs
• What types of safety bugs occur in open-source CPSs?
• Context: Cyber-physical Systems (CPSs)
The COSMOS Project has
received funding from
the European Union’s
Horizon 2020 Research
and Innovation
Programme under grant
agreement No. 957254.
UAV
Thanks for the Attention!
• Any Questions?
J1 presentation:
“Automated Identification and Qualitative
Characterization of Safety Concerns Reported in UAV
Software Platforms”. Transactions on Software
Engineering and Methodology.
Mon 11 - Fri 15 September 2023, Luxembourg
Sebastiano Panichella
Zurich University of Applied Sciences
https://spanichella.github.io/
Link to the tool behind our research
https://github.com/spanichella/SALIENT-TOOL
1 de 27

Recomendados

An Empirical Characterization of Software Bugs in Open-Source Cyber-Physical ... por
An Empirical Characterization of Software Bugs in Open-Source Cyber-Physical ...An Empirical Characterization of Software Bugs in Open-Source Cyber-Physical ...
An Empirical Characterization of Software Bugs in Open-Source Cyber-Physical ...Sebastiano Panichella
52 visualizações35 slides
Testing and Development Challenges for Complex Cyber-Physical Systems: Insi... por
Testing and Development Challenges for  Complex Cyber-Physical Systems:  Insi...Testing and Development Challenges for  Complex Cyber-Physical Systems:  Insi...
Testing and Development Challenges for Complex Cyber-Physical Systems: Insi...Sebastiano Panichella
39 visualizações148 slides
Testing with Fewer Resources: Toward Adaptive Approaches for Cost-effective T... por
Testing with Fewer Resources: Toward Adaptive Approaches for Cost-effective T...Testing with Fewer Resources: Toward Adaptive Approaches for Cost-effective T...
Testing with Fewer Resources: Toward Adaptive Approaches for Cost-effective T...Sebastiano Panichella
253 visualizações168 slides
Personal Computer And Application Software D. por
Personal Computer And Application Software D.Personal Computer And Application Software D.
Personal Computer And Application Software D.Jacqueline Thomas
2 visualizações81 slides
No Safety Without Security por
No Safety Without SecurityNo Safety Without Security
No Safety Without SecuritySecurity Innovation
399 visualizações21 slides
[CB16] Security in the IoT World: Analyzing the Security of Mobile Apps for A... por
[CB16] Security in the IoT World: Analyzing the Security of Mobile Apps for A...[CB16] Security in the IoT World: Analyzing the Security of Mobile Apps for A...
[CB16] Security in the IoT World: Analyzing the Security of Mobile Apps for A...CODE BLUE
735 visualizações52 slides

Mais conteúdo relacionado

Similar a Automated Identification and Qualitative Characterization of Safety Concerns Reported in UAV Software Platforms

Open Source Insight: Who Owns Linux? TRITON Attack, App Security Testing, Fut... por
Open Source Insight: Who Owns Linux? TRITON Attack, App Security Testing, Fut...Open Source Insight: Who Owns Linux? TRITON Attack, App Security Testing, Fut...
Open Source Insight: Who Owns Linux? TRITON Attack, App Security Testing, Fut...Black Duck by Synopsys
450 visualizações17 slides
20140121 cisec-safety criticalsoftwaredevelopment por
20140121 cisec-safety criticalsoftwaredevelopment20140121 cisec-safety criticalsoftwaredevelopment
20140121 cisec-safety criticalsoftwaredevelopmentCISEC
2.2K visualizações41 slides
2014-12-16 defense news - shutdown the hackers por
2014-12-16  defense news - shutdown the hackers2014-12-16  defense news - shutdown the hackers
2014-12-16 defense news - shutdown the hackersShawn Wells
327 visualizações66 slides
PROGRAMMING AND CYBER SECURITY por
PROGRAMMING AND CYBER SECURITYPROGRAMMING AND CYBER SECURITY
PROGRAMMING AND CYBER SECURITYSylvain Martinez
235 visualizações23 slides
Mechanical phish por
Mechanical phishMechanical phish
Mechanical phishDebdeep Banerjee
54 visualizações11 slides
Open Source Insight: Auto Security & Hackers, Killer Robots, & Containers Gon... por
Open Source Insight: Auto Security & Hackers, Killer Robots, & Containers Gon...Open Source Insight: Auto Security & Hackers, Killer Robots, & Containers Gon...
Open Source Insight: Auto Security & Hackers, Killer Robots, & Containers Gon...Black Duck by Synopsys
299 visualizações17 slides

Similar a Automated Identification and Qualitative Characterization of Safety Concerns Reported in UAV Software Platforms(20)

Open Source Insight: Who Owns Linux? TRITON Attack, App Security Testing, Fut... por Black Duck by Synopsys
Open Source Insight: Who Owns Linux? TRITON Attack, App Security Testing, Fut...Open Source Insight: Who Owns Linux? TRITON Attack, App Security Testing, Fut...
Open Source Insight: Who Owns Linux? TRITON Attack, App Security Testing, Fut...
Black Duck by Synopsys450 visualizações
20140121 cisec-safety criticalsoftwaredevelopment por CISEC
20140121 cisec-safety criticalsoftwaredevelopment20140121 cisec-safety criticalsoftwaredevelopment
20140121 cisec-safety criticalsoftwaredevelopment
CISEC2.2K visualizações
2014-12-16 defense news - shutdown the hackers por Shawn Wells
2014-12-16  defense news - shutdown the hackers2014-12-16  defense news - shutdown the hackers
2014-12-16 defense news - shutdown the hackers
Shawn Wells327 visualizações
PROGRAMMING AND CYBER SECURITY por Sylvain Martinez
PROGRAMMING AND CYBER SECURITYPROGRAMMING AND CYBER SECURITY
PROGRAMMING AND CYBER SECURITY
Sylvain Martinez235 visualizações
Mechanical phish por Debdeep Banerjee
Mechanical phishMechanical phish
Mechanical phish
Debdeep Banerjee54 visualizações
Open Source Insight: Auto Security & Hackers, Killer Robots, & Containers Gon... por Black Duck by Synopsys
Open Source Insight: Auto Security & Hackers, Killer Robots, & Containers Gon...Open Source Insight: Auto Security & Hackers, Killer Robots, & Containers Gon...
Open Source Insight: Auto Security & Hackers, Killer Robots, & Containers Gon...
Black Duck by Synopsys299 visualizações
G. Gritsai, A. Timorin, Y. Goltsev, R. Ilin, S. Gordeychik, and A. Karpin, “S... por qqlan
G. Gritsai, A. Timorin, Y. Goltsev, R. Ilin, S. Gordeychik, and A. Karpin, “S...G. Gritsai, A. Timorin, Y. Goltsev, R. Ilin, S. Gordeychik, and A. Karpin, “S...
G. Gritsai, A. Timorin, Y. Goltsev, R. Ilin, S. Gordeychik, and A. Karpin, “S...
qqlan716 visualizações
Machine Learning-Based Prototyping of Graphical User Interfaces for Mobile Apps por Kevin Moran
Machine Learning-Based Prototyping of Graphical User Interfaces for Mobile AppsMachine Learning-Based Prototyping of Graphical User Interfaces for Mobile Apps
Machine Learning-Based Prototyping of Graphical User Interfaces for Mobile Apps
Kevin Moran703 visualizações
Defect Prediction & Prevention In Automotive Software Development por RAKESH RANA
Defect Prediction & Prevention In Automotive Software DevelopmentDefect Prediction & Prevention In Automotive Software Development
Defect Prediction & Prevention In Automotive Software Development
RAKESH RANA827 visualizações
PHDays 8: Vulnerability Databases. Sifting thousands tons of verbal ore por Alexander Leonov
PHDays 8: Vulnerability Databases. Sifting thousands tons of verbal orePHDays 8: Vulnerability Databases. Sifting thousands tons of verbal ore
PHDays 8: Vulnerability Databases. Sifting thousands tons of verbal ore
Alexander Leonov746 visualizações
Intelligence on the Intractable Problem of Software Security por Tyler Shields
Intelligence on the Intractable Problem of Software SecurityIntelligence on the Intractable Problem of Software Security
Intelligence on the Intractable Problem of Software Security
Tyler Shields407 visualizações
Achieving Software Safety, Security, and Reliability Part 1: Common Industry ... por Perforce
Achieving Software Safety, Security, and Reliability Part 1: Common Industry ...Achieving Software Safety, Security, and Reliability Part 1: Common Industry ...
Achieving Software Safety, Security, and Reliability Part 1: Common Industry ...
Perforce140 visualizações
Open Source Insight: Securing IoT, Atlanta Ransomware Attack, Congress on Cyb... por Black Duck by Synopsys
Open Source Insight: Securing IoT, Atlanta Ransomware Attack, Congress on Cyb...Open Source Insight: Securing IoT, Atlanta Ransomware Attack, Congress on Cyb...
Open Source Insight: Securing IoT, Atlanta Ransomware Attack, Congress on Cyb...
Black Duck by Synopsys934 visualizações
Open Source Insight: Hospital, Medical Devices, Banking, and Automotive Cyber... por Black Duck by Synopsys
Open Source Insight: Hospital, Medical Devices, Banking, and Automotive Cyber...Open Source Insight: Hospital, Medical Devices, Banking, and Automotive Cyber...
Open Source Insight: Hospital, Medical Devices, Banking, and Automotive Cyber...
Black Duck by Synopsys289 visualizações
Semi-Automated Security Testing of Web applications por Ram G Athreya
Semi-Automated Security Testing of Web applicationsSemi-Automated Security Testing of Web applications
Semi-Automated Security Testing of Web applications
Ram G Athreya3.6K visualizações
Design reliability 2.0: Safety is Everything por Amir Rahat
Design reliability 2.0: Safety is Everything Design reliability 2.0: Safety is Everything
Design reliability 2.0: Safety is Everything
Amir Rahat533 visualizações
9 th International Conference on Information Technology, Control, Chaos, Mode... por IJCSEA Journal
9 th International Conference on Information Technology, Control, Chaos, Mode...9 th International Conference on Information Technology, Control, Chaos, Mode...
9 th International Conference on Information Technology, Control, Chaos, Mode...
IJCSEA Journal5 visualizações
Exposed! A case study on the vulnerability-proneness of Google Play Apps por Sebastiano Panichella
Exposed! A case study on the vulnerability-proneness of Google Play AppsExposed! A case study on the vulnerability-proneness of Google Play Apps
Exposed! A case study on the vulnerability-proneness of Google Play Apps
Sebastiano Panichella21 visualizações
Semantic Web for Advanced Engineering por Marta Sabou
Semantic Web for Advanced EngineeringSemantic Web for Advanced Engineering
Semantic Web for Advanced Engineering
Marta Sabou739 visualizações
Cyber Security Challenges with IoT, Smart City por Bhaskar Chattoraj
Cyber Security Challenges with IoT, Smart City Cyber Security Challenges with IoT, Smart City
Cyber Security Challenges with IoT, Smart City
Bhaskar Chattoraj2.1K visualizações

Mais de Sebastiano Panichella

The 2nd Intl. Workshop on NL-based Software Engineering por
The 2nd Intl. Workshop on NL-based Software EngineeringThe 2nd Intl. Workshop on NL-based Software Engineering
The 2nd Intl. Workshop on NL-based Software EngineeringSebastiano Panichella
926 visualizações31 slides
The 16th Intl. Workshop on Search-Based and Fuzz Testing por
The 16th Intl. Workshop on Search-Based and Fuzz TestingThe 16th Intl. Workshop on Search-Based and Fuzz Testing
The 16th Intl. Workshop on Search-Based and Fuzz TestingSebastiano Panichella
2.4K visualizações37 slides
Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Nei... por
Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Nei...Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Nei...
Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Nei...Sebastiano Panichella
41 visualizações22 slides
COSMOS: DevOps for complex cyber-physical systems (H2020 Project) - WASOS wor... por
COSMOS: DevOps for complex cyber-physical systems (H2020 Project) - WASOS wor...COSMOS: DevOps for complex cyber-physical systems (H2020 Project) - WASOS wor...
COSMOS: DevOps for complex cyber-physical systems (H2020 Project) - WASOS wor...Sebastiano Panichella
127 visualizações45 slides
Search-based Software Testing (SBST) '22 por
Search-based Software Testing (SBST) '22Search-based Software Testing (SBST) '22
Search-based Software Testing (SBST) '22Sebastiano Panichella
299 visualizações25 slides
NLBSE’22: Tool Competition por
NLBSE’22: Tool CompetitionNLBSE’22: Tool Competition
NLBSE’22: Tool CompetitionSebastiano Panichella
171 visualizações17 slides

Mais de Sebastiano Panichella(20)

The 2nd Intl. Workshop on NL-based Software Engineering por Sebastiano Panichella
The 2nd Intl. Workshop on NL-based Software EngineeringThe 2nd Intl. Workshop on NL-based Software Engineering
The 2nd Intl. Workshop on NL-based Software Engineering
Sebastiano Panichella926 visualizações
The 16th Intl. Workshop on Search-Based and Fuzz Testing por Sebastiano Panichella
The 16th Intl. Workshop on Search-Based and Fuzz TestingThe 16th Intl. Workshop on Search-Based and Fuzz Testing
The 16th Intl. Workshop on Search-Based and Fuzz Testing
Sebastiano Panichella2.4K visualizações
Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Nei... por Sebastiano Panichella
Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Nei...Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Nei...
Simulation-based Test Case Generation for Unmanned Aerial Vehicles in the Nei...
Sebastiano Panichella41 visualizações
COSMOS: DevOps for complex cyber-physical systems (H2020 Project) - WASOS wor... por Sebastiano Panichella
COSMOS: DevOps for complex cyber-physical systems (H2020 Project) - WASOS wor...COSMOS: DevOps for complex cyber-physical systems (H2020 Project) - WASOS wor...
COSMOS: DevOps for complex cyber-physical systems (H2020 Project) - WASOS wor...
Sebastiano Panichella127 visualizações
Search-based Software Testing (SBST) '22 por Sebastiano Panichella
Search-based Software Testing (SBST) '22Search-based Software Testing (SBST) '22
Search-based Software Testing (SBST) '22
Sebastiano Panichella299 visualizações
NLBSE’22: Tool Competition por Sebastiano Panichella
NLBSE’22: Tool CompetitionNLBSE’22: Tool Competition
NLBSE’22: Tool Competition
Sebastiano Panichella171 visualizações
"An NLP-based Tool for Software Artifacts Analysis" at @ICSME2021. por Sebastiano Panichella
 "An NLP-based Tool for Software Artifacts Analysis" at @ICSME2021.  "An NLP-based Tool for Software Artifacts Analysis" at @ICSME2021.
"An NLP-based Tool for Software Artifacts Analysis" at @ICSME2021.
Sebastiano Panichella468 visualizações
An Empirical Investigation of Relevant Changes and Automation Needs in Modern... por Sebastiano Panichella
An Empirical Investigation of Relevant Changes and Automation Needs in Modern...An Empirical Investigation of Relevant Changes and Automation Needs in Modern...
An Empirical Investigation of Relevant Changes and Automation Needs in Modern...
Sebastiano Panichella402 visualizações
Search-Based Software Testing Tool Competition 2021 by Sebastiano Panichella,... por Sebastiano Panichella
Search-Based Software Testing Tool Competition 2021 by Sebastiano Panichella,...Search-Based Software Testing Tool Competition 2021 by Sebastiano Panichella,...
Search-Based Software Testing Tool Competition 2021 by Sebastiano Panichella,...
Sebastiano Panichella402 visualizações
A Framework for Multi-source Studies based on Unstructured Data. por Sebastiano Panichella
A Framework for Multi-source Studies based on Unstructured Data.A Framework for Multi-source Studies based on Unstructured Data.
A Framework for Multi-source Studies based on Unstructured Data.
Sebastiano Panichella78 visualizações
Revisiting Test Smells in Automatically Generated Tests: Limitations, Pitfall... por Sebastiano Panichella
Revisiting Test Smells in Automatically Generated Tests: Limitations, Pitfall...Revisiting Test Smells in Automatically Generated Tests: Limitations, Pitfall...
Revisiting Test Smells in Automatically Generated Tests: Limitations, Pitfall...
Sebastiano Panichella524 visualizações
Requirements-Collector: Automating Requirements Specification from Elicitatio... por Sebastiano Panichella
Requirements-Collector: Automating Requirements Specification from Elicitatio...Requirements-Collector: Automating Requirements Specification from Elicitatio...
Requirements-Collector: Automating Requirements Specification from Elicitatio...
Sebastiano Panichella296 visualizações
Unit Testing Tool Competition-Eighth Round por Sebastiano Panichella
Unit Testing Tool Competition-Eighth RoundUnit Testing Tool Competition-Eighth Round
Unit Testing Tool Competition-Eighth Round
Sebastiano Panichella195 visualizações
Cultural Exchange - ICSE 2020 por Sebastiano Panichella
Cultural Exchange - ICSE 2020Cultural Exchange - ICSE 2020
Cultural Exchange - ICSE 2020
Sebastiano Panichella528 visualizações
Testing with Fewer Resources: An Adaptive Approach to Performance-Aware Test ... por Sebastiano Panichella
Testing with Fewer Resources: An Adaptive Approach to Performance-Aware Test ...Testing with Fewer Resources: An Adaptive Approach to Performance-Aware Test ...
Testing with Fewer Resources: An Adaptive Approach to Performance-Aware Test ...
Sebastiano Panichella250 visualizações
A Mixed Graph-Relational Dataset of Socio-technical interactions in Open Sour... por Sebastiano Panichella
A Mixed Graph-Relational Dataset of Socio-technical interactions in Open Sour...A Mixed Graph-Relational Dataset of Socio-technical interactions in Open Sour...
A Mixed Graph-Relational Dataset of Socio-technical interactions in Open Sour...
Sebastiano Panichella166 visualizações
Action-based Recommendation in Pull-request Development por Sebastiano Panichella
Action-based Recommendation in Pull-request DevelopmentAction-based Recommendation in Pull-request Development
Action-based Recommendation in Pull-request Development
Sebastiano Panichella148 visualizações
Ticket Tagger: Machine Learning Driven Issue Classification por Sebastiano Panichella
Ticket Tagger: Machine Learning Driven Issue ClassificationTicket Tagger: Machine Learning Driven Issue Classification
Ticket Tagger: Machine Learning Driven Issue Classification
Sebastiano Panichella692 visualizações
DRONE: A Tool to Detect and Repair Directive Defects in Java APIs Documentation por Sebastiano Panichella
DRONE: A Tool to Detect and Repair Directive Defects in Java APIs DocumentationDRONE: A Tool to Detect and Repair Directive Defects in Java APIs Documentation
DRONE: A Tool to Detect and Repair Directive Defects in Java APIs Documentation
Sebastiano Panichella447 visualizações
The Cloudification Perspectives of Search-based Software Testing por Sebastiano Panichella
The Cloudification Perspectives of Search-based Software TestingThe Cloudification Perspectives of Search-based Software Testing
The Cloudification Perspectives of Search-based Software Testing
Sebastiano Panichella475 visualizações

Último

Out-of-Market Efficiencies in Competition Enforcement – ROSE – December 2023 ... por
Out-of-Market Efficiencies in Competition Enforcement – ROSE – December 2023 ...Out-of-Market Efficiencies in Competition Enforcement – ROSE – December 2023 ...
Out-of-Market Efficiencies in Competition Enforcement – ROSE – December 2023 ...OECD Directorate for Financial and Enterprise Affairs
59 visualizações6 slides
Alternatives to Leniency Programmes – SERBIA – December 2023 OECD discussion por
Alternatives to Leniency Programmes – SERBIA – December 2023 OECD discussionAlternatives to Leniency Programmes – SERBIA – December 2023 OECD discussion
Alternatives to Leniency Programmes – SERBIA – December 2023 OECD discussionOECD Directorate for Financial and Enterprise Affairs
43 visualizações5 slides
Use of Economic Evidence in Cartel Cases – DAVIES – December 2023 OECD discus... por
Use of Economic Evidence in Cartel Cases – DAVIES – December 2023 OECD discus...Use of Economic Evidence in Cartel Cases – DAVIES – December 2023 OECD discus...
Use of Economic Evidence in Cartel Cases – DAVIES – December 2023 OECD discus...OECD Directorate for Financial and Enterprise Affairs
78 visualizações38 slides
Effective Speaking (Chapter 3).pdf por
Effective Speaking (Chapter 3).pdfEffective Speaking (Chapter 3).pdf
Effective Speaking (Chapter 3).pdfInstitute of Hotel Management, Hajipur, Patna, Bihar.
5 visualizações13 slides
Serial Acquisitions and Industry Roll-ups – KOKKORIS – December 2023 OECD dis... por
Serial Acquisitions and Industry Roll-ups – KOKKORIS – December 2023 OECD dis...Serial Acquisitions and Industry Roll-ups – KOKKORIS – December 2023 OECD dis...
Serial Acquisitions and Industry Roll-ups – KOKKORIS – December 2023 OECD dis...OECD Directorate for Financial and Enterprise Affairs
140 visualizações8 slides
Ex-post Assessment of Merger Remedies – OECD – December 2023 OECD discussion por
Ex-post Assessment of Merger Remedies – OECD – December 2023 OECD discussionEx-post Assessment of Merger Remedies – OECD – December 2023 OECD discussion
Ex-post Assessment of Merger Remedies – OECD – December 2023 OECD discussionOECD Directorate for Financial and Enterprise Affairs
46 visualizações7 slides

Último(20)

ORAL PRESENTATION por alghalakhalid15
ORAL PRESENTATIONORAL PRESENTATION
ORAL PRESENTATION
alghalakhalid155 visualizações
PB CV v0.4 por Pedro Borracha
PB CV v0.4PB CV v0.4
PB CV v0.4
Pedro Borracha8 visualizações
oral presentation .pdf por HindAlmesmary
oral presentation .pdforal presentation .pdf
oral presentation .pdf
HindAlmesmary8 visualizações
RTC2023_Boost-App-Integration-with-AI_Kim.pdf por hossenkamal2
RTC2023_Boost-App-Integration-with-AI_Kim.pdfRTC2023_Boost-App-Integration-with-AI_Kim.pdf
RTC2023_Boost-App-Integration-with-AI_Kim.pdf
hossenkamal28 visualizações
PRESENTATION.pptx por yunuskhan558800
PRESENTATION.pptxPRESENTATION.pptx
PRESENTATION.pptx
yunuskhan5588007 visualizações
NguyenChristine_Portfolio (1).pdf por chnguyentv9
NguyenChristine_Portfolio (1).pdfNguyenChristine_Portfolio (1).pdf
NguyenChristine_Portfolio (1).pdf
chnguyentv930 visualizações
تنزيل (1).pdf DVT por taalali1
تنزيل (1).pdf  DVT تنزيل (1).pdf  DVT
تنزيل (1).pdf DVT
taalali18 visualizações
corporate-presentation.pdf por Shaun Heinrichs
corporate-presentation.pdfcorporate-presentation.pdf
corporate-presentation.pdf
Shaun Heinrichs135 visualizações
Cafeteria-Blog 41. por Hollywood Actress
Cafeteria-Blog 41.Cafeteria-Blog 41.
Cafeteria-Blog 41.
Hollywood Actress5 visualizações

Automated Identification and Qualitative Characterization of Safety Concerns Reported in UAV Software Platforms

  • 1. J1 presentation: “Automated Identification and Qualitative Characterization of Safety Concerns Reported in UAV Software Platforms” Transactions on Software Engineering and Methodology. Mon 11 - Fri 15 September 2023, Luxembourg Sebastiano Panichella Zurich University of Applied Sciences https://spanichella.github.io/ Andrea Di Sorbo Fiorella Zampetti Aaron Visaggio Massimiliano Di Penta University of Sannio IEEE/ACM International Conference on Automated Software Engineering (ASE 2023)
  • 2. Outline 2 • DevOps shortcomings for Complex CPSs • What types of safety bugs occur in open-source CPSs? • Context: Cyber-physical Systems (CPSs) The COSMOS Project has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under grant agreement No. 957254. UAV
  • 3. “Emerging Cyber-physical Systems (CPS) will play a crucial role in the quality of life of European citizens and the future of the European economy” Context • CPS relevant sectors: • Healthcare • Automotive • Water Monitoring • Railway • Manufacturing • Avionics • etc. MEDICAL DELIVERY FOOD DELIVERY • Avionics 3
  • 5. Background First aerodynamic flight on another planet. Landed with Perseverance rover on 18 February 2021 SPACE EXPLORATION
  • 6. • - • Our (Software Engineering) view of DevOps and AI for IoT systems: • DevOps and Continuous Delivery (CD): Whats is it? • Present, Challenges, and Opportunities • Relevant Research Questions • Artificial Intelligence (AI) and Testing Automation: • Present, Challenges, and Opportunities • User-oriented Testing Automation • Relevant Research Questions “We all recognize the relevance and capacity of contemporary cyber- physical systems for building the future of our society, but ongoing research in the field is also clearly failing in making the right countermeasures to avoid that CPS usage affects human being safety”. In “Self-driving Uber kills Arizona woman in first fatal crash involving pedestrian” “Swiss Post drone crashes in Zurich Problem Statement “A simple software update was the direct cause of the fatal crashes of the Boeing 737” 6
  • 7. • - • Our (Software Engineering) view of DevOps and AI for IoT systems: • DevOps and Continuous Delivery (CD): Whats is it? • Present, Challenges, and Opportunities • Relevant Research Questions • Artificial Intelligence (AI) and Testing Automation: • Present, Challenges, and Opportunities • User-oriented Testing Automation • Relevant Research Questions “Self-driving Uber kills Arizona woman in first fatal crash involving pedestrian” “Swiss Post drone crashes in Zurich Challenge “A simple software update was the direct cause of the fatal crashes of the Boeing 737” Observability, testability, and predictability of the behavior of emerging CPS is highly limited and, unfortunately, their usage in the real world can lead to fatal crashes sometimes tragically involving also humans 7
  • 8. Sebastiano Panichella Sajad Khatiri Christian Birchler COSMOS: DevOps for Complex Cyber-physical Systems https://www.cosmos-devops.org/ https://twitter.com/COSMOS_DEVOPS https://lnkd.in/eUVeaYaz
  • 9. 9 What types of bugs occur in open-source CPSs? CPS Bugs
  • 10. 10 Famous Software Failure in a CPS: The Case of Ariane 5
  • 11. 11 Bugs and Failures in CPS CPS Bug is “a flaw in the hardware (not properly handled by the software), or an incorrect interaction between the software and hardware components leading to a CPS misbehavior’’ A CPS bug can manifest as a CPS failure, which makes a CPS unable to deliver its required functionality or not fulfilling some non-functional properties Properties
  • 12. 12 Bugs in the PX4 Project https://github.com/PX4/PX4-Autopilot/issues/8980 Px4 Issue 8980: Unsuccessful flight “ Autopilot receiving noisy sensor-data…“
  • 13. 13 Fiorella Zampetti, Ritu Kapur, Massimiliano Di Penta, Sebastiano Panichella: An Empirical Characterization of Software Bugs in Open-Source Cyber-Physical Systems. Journal of Systems & Software (JSS). What types of bugs occur in Open-source CPSs? CPS Bugs 1,151 closed issues sampled from 14 open-source CPS projects CPS bugs taxonomy comprises 8 different high-level categories
  • 14. 17 What about Safety Bugs of CPSs? CPS Bugs
  • 15. 18 What are the main Hazards and Accidents Emerging from Safety Issues Reported in UAV Software Platforms? Andrea Di Sorbo, Fiorella Zampetti, Corrado A. Visaggio, Massimiliano Di Penta, and Sebastiano Panichella: Automated Identification and Qualitative Characterization of Safety Concerns Reported in UAV Software Platforms. Transactions on Software Engineering and Methodology. 150 Projects
  • 16. 19 What are the main Hazards and Accidents Emerging from Safety Issues Reported in UAV Software Platforms? CPS Safety Related Issues of UAVs Andrea Di Sorbo, Fiorella Zampetti, Corrado A. Visaggio, Massimiliano Di Penta, and Sebastiano Panichella: Automated Identification and Qualitative Characterization of Safety Concerns Reported in UAV Software Platforms. Transactions on Software Engineering and Methodology.
  • 17. 20 RQ1: To what extent can machine learning models automatically identify safety-related concerns in issue reports of UAV software platforms? RQ2: What are the main hazards and accidents emerging from safety issues reported in UAV software platforms? CPS Safety Related Issues of UAVs RQ1: To what extent can machine learning models automatically identify safety-related concerns in issue reports of UAV software platforms? RQ2: What are the main hazards and accidents emerging from safety issues reported in UAV software platforms? What are the main Hazards and Accidents Emerging from Safety Issues Reported in UAV Software Platforms?
  • 18. 21 Gold Standard Definition 1,916 sentences Cohen’s k = 0.68 (substantial) 837 (43.7%) safety 1,079 (56.3%) non-safety
  • 19. 22 Safety Bugs in open-source CPSs Preprocessing of CPS Safety Related Issues Tf Matrix GitHub Issues Tf Matrix Tf-idf Matrix RQ1: To what extent can machine learning models automatically identify safety-related concerns in issue reports of UAV software platforms? RQ2: What are the main hazards and accidents emerging from safety issues reported in UAV software platforms?
  • 20. 23 Safety Bugs in open-source CPSs ML prediction & Hyperparameter optimization Tf Matrix Tf-idf Matrix RQ1: To what extent can machine learning models automatically identify safety-related concerns in issue reports of UAV software platforms? RQ2: What are the main hazards and accidents emerging from safety issues reported in UAV software platforms?
  • 21. 24 Safety Bugs in open-source CPSs Performance of the ML classifiers for classifying safety-related issues in pull requests Tf-idf wins over other processing approaches > 80% Precision, Recall, and F-measure (10- fold validation) using word embeddings with FastText Link to the tool behind our research https://github.com/spanichella/SALIENT-TOOL
  • 22. 26 What are the main Hazards and Accidents Emerging from Safety Issues Reported in UAV Software Platforms? 273 safety-related issues and PR 19 Hazard categories 7 Accident categories
  • 23. 27 Co-occurrences of hazard categories and accident categories Hazard Accident Hazard categories and corresponding occurrences in our dataset of 273 safety-related issues and pull requests. What are the main Hazards and Accidents Emerging from Safety Issues Reported in UAV Software Platforms?
  • 24. What are the main Hazards and Accidents Emerging from Safety Issues Reported in UAV Software Platforms?
  • 25. 29 Co-occurrences of hazard categories and accident categories Hazard Accident Hazard categories and corresponding occurrences in our dataset of 273 safety-related issues and pull requests. Main Findings: We need automated approaches to monitor and address safety concerns within a CPS DevOps pipeline What are the main Hazards and Accidents Emerging from Safety Issues Reported in UAV Software Platforms?
  • 26. Summary 30 • DevOps shortcomings for Complex CPSs • What types of safety bugs occur in open-source CPSs? • Context: Cyber-physical Systems (CPSs) The COSMOS Project has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under grant agreement No. 957254. UAV
  • 27. Thanks for the Attention! • Any Questions? J1 presentation: “Automated Identification and Qualitative Characterization of Safety Concerns Reported in UAV Software Platforms”. Transactions on Software Engineering and Methodology. Mon 11 - Fri 15 September 2023, Luxembourg Sebastiano Panichella Zurich University of Applied Sciences https://spanichella.github.io/ Link to the tool behind our research https://github.com/spanichella/SALIENT-TOOL