The document summarizes the findings of a qualitative study comparing performance bugs to non-performance bugs in the Chrome and Firefox browsers. The study analyzed hundreds of bug reports to identify differences across four dimensions: impact on the user, context of the bug, discussion around fixing the bug, and the actual bug fix. The study found that performance bugs generally have a greater negative impact, are more complex to fix, require more discussion and collaboration, and are more likely to block software releases.
7. Security versus Performance Bugs:
A Case Study on
Shahed&Zaman,&Bram&Adams,&Ahmed&E.&Hassan
SoUware&Analysis&and&Intelligence&Lab&(SAIL),&Queen’s&University
6
8. MSR%2011%Quan4ta4ve%Study
Time People Bug%Fix
&&&&&Performance&bugs&take& Performance&bugs&are&fixed& Performance&bug&fixes&
more%4me&to&fix by&more&experienced& are&more&complex
developers
7
40. MSR%2011%Quan4ta4ve%Study%(reprise)
Performance&bugs&are&fixed&
&&&&&Performance&bugs&take& Performance&bug&fixes&
by&more&experienced&
more%4me&to&fix are&more&complex
developers
replication problems more release
blocking
more dependencies
m ore discussion users leaving
WFM after a long time 36
41. Threats to Validity
‘slow’,%‘perf’,%
‘hang’
human&bias
heurisEcs&for&
only&4&
only&2&subject performance&bug&
dimensions
systems idenEficaEon
&&&20&subD
dimensions 37
42. Study Setup
Chrome ®
no traceability
295,198%bugs context&of&the&bug
44,997%bugs
7,603%performance%bugs 510%performance%bugs
4 Dimensions
(20 subdimensions)
bug&fix&validaEon
impact&on&the&user
100%Bugs 100%Bugs
+ +
100%Bugs 100%Bugs
= =
200%Bugs 200%Bugs
Performance%Bugs NonEperformance%bugs
bug&fix
is®ression + + MSR%2011%Quan4ta4ve%Study%(reprise)
blocks&release + +
WFM&aUer&a&long&Eme + + Performance&bugs&are&fixed&
&&&&&Performance&bugs&take& Performance&bug&fixes&
impact&on&the&user by&more&experienced&
scares&users&away + + more%4me&to&fix are&more&complex
developers
has&measurements + +
has&test&cases + +
contains&stacktrace
no reproducability + +
discussion!
has&replicaEon&problems + +
? + replication problems more release
is&reported&by&a&developer
context&of&the&bug
is&a&duplicate E E blocking
more dependencies
has&discussion + +
depends&on&other&bug + + more discussion users leaving
bug&fix
blocks&other&bug + E WFM after a long time 37