Fuxiang's FSE 2015 talk

1. Crowd
Debugging
Fuxiang CHEN Sunghun KIM
1ESEC/FSE 2015 (BERGAMO, ITALY, Aug 30 – Sep 4)

2. Debugging is
Hard
2
[ICSE ‘13 Nguyen]
•Sig. amt. of time
•Root cause found, fixing difficult
[ESEC/FSE ’09 Ashok]
•Sig. amt. of time
•Similar bugs debugged
What’s wrong
with my code?

3. Some Static Analysis tools you can try
- faster/cheaper than manual inspection
3

4. But….. they are
underusedThey are known for..
1.producing many warnings …. AND
2.Many of them are false positives…. AND
3.Ways In which warnings are presented
(counterintuitive, confusion)
4
[ICSE ‘13 Johnson]

5. Another way to debug is using
Stack Overflow
5

6. 6

7. Oh my god….. This piece of
code looks so similar to
mine..
7
So, what does it means by many
recurring questions?
Many developers are facing many similar issues…
Challenges for developers?
• Needs frequent visit
• Rapid growth of crowd knowledge
• May not even know you have problem

8. 8

9. Overview of Approach
9
Code-Pairs
Database
Code-Pairs
Database
Code Clone
Detection
Code Clone
Detection
FilteringFiltering

10. Observation: Same Code Element in Question &
Answer
10
Stack Overflow Question Code Fragment
Stack Overflow Answer Code Fragment

11. Observation: Same Code Element in Title & Question
11
Stack Overflow Question Title
Stack Overflow Question Code Fragment

12. Code-Pairs
Database
Code-Pairs
Database
Question TitleQuestion Title
1
Code-Pairs
Database
Code-Pairs
Database
Code Clone
Detection
Code Clone
Detection
FilteringFiltering

13. 13
Question
Code
Database
Question
Code
Database
Target CodeTarget Code
SolrXMLSerializer.java (lucene)
Code-Pairs
Database
Code-Pairs
Database
Code Clone
Detection
Code Clone
Detection
FilteringFiltering

14. Question
Code
Database
Question
Code
Database
14
Searching
Similar Code
CCFinderX
30 tokens
Code-Pairs
Database
Code-Pairs
Database
Code Clone
Detection
Code Clone
Detection
FilteringFiltering

15. Question
Code
Database
Question
Code
Database
15
Found!
Code-Pairs
Database
Code-Pairs
Database
Code Clone
Detection
Code Clone
Detection
FilteringFiltering
Stack Overflow Question Code Fragment
Target Code Fragment

16. 16
Code-Pairs
Database
Code-Pairs
Database
Code Clone
Detection
Code Clone
Detection
FilteringFiltering
Title &
Element
Filters
Title &
Element
Filters
Non-
Essential
Code Filters
Non-
Essential
Code Filters
Similarities
Filters
Similarities
Filters

17. 17
Title & Element
Filters
Title & Element
Filters
Non-Essential Code
Filters
Non-Essential Code
Filters
Similarities FiltersSimilarities Filters
Stack Overflow Question Title
Target Clone

18. 18
Title & Element
Filters
Title & Element
Filters
Non-Essential Code
Filters
Non-Essential Code
Filters
Similarities FiltersSimilarities Filters
Target Clone
Stack Overflow Answer Code Fragment

19. • [ICSE ‘11 Kawrykow]
• Non-essential code-changes
• Can cause inaccurate representation of software development effort
19
Title & Element
Filters
Title & Element
Filters
Non-Essential Code
Filters
Non-Essential Code
Filters
Similarities FiltersSimilarities Filters

20. • SO Clone < 4
• Overridden methods
• Single Statement Block
20
Title & Element
Filters
Title & Element
Filters
Non-Essential Code
Filters
Non-Essential Code
Filters
Similarities FiltersSimilarities Filters

21. 21
Similarity Filter #1
•SO Clone/Answer Ratio > 0.8
Similarity Filter #2
•SO Clone/Answer Ratio > 0.37
•Target Clone/Answer Ratio > 0.25
Title & Element
Filters
Title & Element
Filters
Non-Essential Code
Filters
Non-Essential Code
Filters
Similarities FiltersSimilarities Filters

22. Overview of Approach
22
Code-Pairs
Database
Code-Pairs
Database
Code Clone
Detection
Code Clone
Detection
FilteringFiltering
Title &
Element
Filters
Title &
Element
Filters
Non-
Essential
Code Filters
Non-
Essential
Code Filters
Similarities
Filters
Similarities
Filters

23. 23
Subject Version # Files K LOC
commons-lang 3.4-snapshot 274 63
JFreeChart 1.0.19 654 96
joda-time 2.4 315 80
JStock 1.0.7r 290 50
JStudyPlanner 1.0 34 3
JStudyPlanner 2 2.0 56 4
log4j 2.0.1 905 61
lucene 4.9.0 4826 684

24. RQ1: (Detectability) How many warnings
can be
detected using our technique?
24
RQ2: (Confirmed Bugs) How many warnings from
our technique are confirmed as bugs by developers?
RQ3: (Comparison)
1) How many confirmed bugs can also be detected in FindBugs,
JLint and PMD, &
2) How many bugs are missing which can be found in FindBugs,
JLint and PMD??

25. RQ1: (Detectability) How many warnings
can be
detected using our technique?
189 25

26. 26
RQ2: (Confirmed Bugs) How many warnings from
our technique are confirmed as bugs by developers?

27. 27
Defective Code : AxisEntity.java (Line 139 – 145)
Explanation : Using instanceof for comparison is asymmetric
Suggestion : Use getClass() instead of instanceof
Explanation : getClass() is asymmetric
Reference URL : http://www.stackoverflow.com/questions/7132649
Email Bug Report

28. Subject # of warnings # of reported
warnings
# of confirmed
bugs
# of rejected
warnings
commons-lang 3 0 0 0
JFreeChart 152 152 152 0
joda-time 0 0 0 0
JStock 2 1 1 0
JStudyPlanner 19 18 18 0
JStudyPlanner
2
1 0 0 0
log4j 2 0 0 0
lucene 10 10 0 1
Total 189 181 171 1
28

29. 29
JFreeChart
JStudyPlanner
Thanks for the feedback. I agree with your suggestion and will fix these cases…
- David Gilbert (Project Leader)
Thank you very much for your message. This is very valuable to me… Once again your
research is very helpful. I’ll do all the checks in service objects to handle
NullPointerExceptions.
- Oleg Lukin (Project Owner)

30. RQ3: (Comparison)
1) How many bugs can also be detected in FindBugs, JLint and
PMD, &
30

31. RQ3: (Comparison)
2) How many bugs are missing which can be found in FindBugs, JLint
and PMD?
31
3

32. LIMITATIONS AND THREATS TO
VALIDITY
• Data Type Insensitive
• Code Terms outside Code Blocks
• Evaluation of warnings from other Static Analysis tools
32

33. • New way to debug your code
• Find bugs within well maintained open source code
• Locates and explains the bug
• Provide the crowd’s solution with explanation.
33
Key Takeaways