Presented at CSCW 2011. Joint work with Philip J Guo, Nachiappan Nagappan, and Brendan Murphy.

1. Not my Bug!
Reasons for Software Bug Report Reassignments.
Philip J. Guo
Stanford University
Thomas Zimmermann
Nachiappan Nagappan
Brendan Murphy
Microsoft Research
© Microsoft Corporation

2. A confession…
© Microsoft Corporation

3. Opinions expressed on this slide are the personal opinions of the presenter, not of Microsoft. ;-)
© Microsoft Corporation

4. Backyard Safari Bug Vacuum Best Ever Bug Jar
Price: $19.82 Price: $8.12
Big Bunch O' Bugs
Price: $8.25
Watch a Bug
Price: $7.59
Found viaCorporation
© Microsoft

5. © Microsoft Corporation

6. © Microsoft Corporation

7. Steps to
reproduce
Bug type
How
found?
Assignee
Severity
© Microsoft Corporation

8. BUG TRACKING 101
© Microsoft Corporation

9. Your software fails…
© Microsoft Corporation

10. Slide provided by Sascha
© Microsoft Corporation Just

11. © Microsoft Corporation Picture on the left provided by Rahul Premraj

12. Bugs with reassignments are more
likely to be fixed!
Windows Vista
© Microsoft Corporation

13. BUG REPORT
REASSIGNMENTS
© Microsoft Corporation

14. Methodology
Qualitative • “In your experience, what are reasons why a
bug would be reassigned multiple times”
survey • 358 out of 1,773 responded. Card sort.
Quantitative • All bug reports for Windows Vista
• Logistic regression model for bugs with
analysis “excessive” reassignments
Manual • Random sample of bugs with “excessive”
reassignments (more than 5)
inspection • 50 bug reports
Follow-up • Reassignment patterns, bug pong
survey • 118 out of 397 responded
© Microsoft Corporation

15. What are reasons
for bug report
reassignments?
© Microsoft Corporation

16. #1: Finding the root cause
Root cause in different component. “Bugs many
times are exposed in the UI [user interface], but
are not caused by the team writing the UI code. “
Not enough domain knowledge.
“The filer either lacked the expertise, will, or
time to investigate deep enough to
understand the issue at hand.”
Laziness. “People are willing to do just enough
to convince themselves it isn’t their problem
and then reassign to the person who they think
is closer to the right owner.”
© Microsoft Corporation

17. #2: Determining ownership
Unclear ownership. “The bug falls into an area between
two teams. Say, the USB team and the WPD (Windows
Portable Devices) team. The bug gets kicked around many
times while the teams decide who is actually at fault.”
Found in a bug report: “Dunno who gets this
one, but it’s not me. I don’t have anything to
do with this Component, as far as I know.”
“Playing bug pong between teams
who don’t agree on ownership.”
© Microsoft Corporation

18. #2: Determining ownership
Bug pong /
Hot potato
• Majority of respondents in follow-up survey replied that bug
pong is “uncommon”.
• However, in some situations bug pong occurs frequently:
– for components shared by multiple teams, high in the system
stack, or with unclear ownership;
– near milestones;
– for bugs with incomplete steps to reproduce.
© Microsoft Corporation

19. #3: Poor bug report quality
“In my experience, the most important factor in
multiple reassignments is unclear bug reports.
If the person assigned to the bug doesn’t
understand the issue, they will either assign it
back to the person who opened it, or (rarely,
but it happens) assign it to the wrong person
based on misunderstood information, and then
it will become even worse.”
© Microsoft Corporation

20. #4: Determining proper fix
“There can be multiple possible fixes for
a given issue which can straddle teams,
so the bug can bounce back and forth
until the bug fix strategy is solidified.”
© Microsoft Corporation

21. #5: Workload balancing
“Once the bug has found the right team,
the biggest factor in reassigning is often
load balancing issues across team
members to drive down totals.
As bug counts become more important,
we’ll move issues around frequently.”
© Microsoft Corporation

22. Descriptive statistical analysis
• All pre- and post-release bug reports for
Windows Vista until July 2009.
• Logistic regression model for bug reports with
excessive numbers of reassignments
– We defined “excessive” as greater than 5
(90% of bugs had 5 or fewer reassignments)
• Confirmed qualitative findings
– Details see paper
© Microsoft Corporation

23. Quantifying reassignment cycles
First assignee Last assignee
Cycle at the beginning
First assignee Last assignee
Cycle at the end
© Microsoft Corporation

24. Quantifying reassignment cycles
Bug reports with reassignment cycles in the
beginning are more likely to be fixed.
Cycle size Beginning End
2 1.11x 0.96x
3 1.10x 0.96x
4 1.12x 0.93x
5 1.04x 0.89x
6 1.07x 0.97x
7 1.03x 0.88x
x is the base probability of any Windows Vista bug
being successfully fixed
© Microsoft Corporation

25. Cycles at the beginning
• Search for correct owner
• Solicit additional information
“The initial bug report is incomplete or inaccurate and
Alice sends back to the tester (Bob) for more information,
better repro steps, etc. This is a common cycle. Once the
bug is improved, it has a high likelihood of being fixed.”
© Microsoft Corporation

26. Cycles at the end
• Discussion on whether bug should be fixed
“This example feels more like a triage cycle where Alice is the PM
[program manager] (or opener) and Bob is the war team/triage
team, etc. The war team is sending the bug back to PM/opener for
justification why the bug should be fixed (and not punted). The fact
that this conversation is happening at all means the bug is at risk
and likely to be punted.”
• Unclear ownership
“When ABA is at the end, I think the bug is likely going back and
forth between two developers, who either do not agree, or do not
want the responsibility of fixing the bug.”
© Microsoft Corporation

27. LESSONS LEARNED
© Microsoft Corporation

28. Reassignments are part of bug fixing
• Bug fixing is a highly collaborative process
that involves many people.
• Not all reassignments are bad.
– Contrary to common belief in software engineering
– Follow-up survey: developers considered only 17.6%
of reassignments to be detrimental (median 10%)
© Microsoft Corporation

29. Finding root causes and owners
• Most reassignments are related to finding
root causes and people.
– Bug triagers acting as information hubs can
help to reduce unneeded reassignments.
• Better tools for finding code ownership and
expertise are needed.
© Microsoft Corporation

30. Fluid bug report assignment
• Give developers a proactive role.
– For example, show developers a list of bug
reports that are related to them and let them pick.
• Assign bug reports to multiple developers
rather than just individuals.
• Assign bug reports to arbitrary artifacts
– Current bug tracking systems allow only
assignment to people and components.
© Microsoft Corporation

31. Awareness and coordination
• Increase the awareness of the changes
happening around bug (re)assignments.
• Visualizations of reassignment pattern.
• Recommender systems based on past
assignment history.
© Microsoft Corporation

32. Conclusions
• Reassignments are beneficial to find the
best person to fix a bug.
• Excessive reassignments are harmful.
• Primary reasons for reassignments:
– finding the root cause, determining ownership,
poor bug report quality, hard to determine
proper fix, and workload balancing.
• Role of reassignments changes over time
© Microsoft Corporation