WCRE2011

1. An
Exploratory
Study
of
the
Evolu4on
of
Communicated
Informa4on
about
the
Execu4on
of
Large
So=ware
Systems
Weiyi Shang
Zhen Ming Jiang
Bram Adams
Ahmed E. Hassan
Michael W. Godfrey
University of Waterloo
Queen’s University
Mohamed Nasser
Parminder Flora
Research In Motion (RIM)

2. 2
What
run-­‐4me
ac4ons
cause
the
failure?

3. Automated
profiling
&
instrumenta4on
Detail
No
domain
knowledge
Large
scale
3

4. Communicated
informa4on
(CI)
Execu4on
Logs
System
Alerts
Code
Comments
/*
…
*/
Sta*c
Dynamic
4
Field
experience
Developer
experience

5. CI
forms
basis
of
Ecosystem
of
Log
Processing
Apps
Workload
recovery
Anomaly
detec4on
Capacity
planning
System
monitoring
Performance
analysis
5
Failure
diagnosis

6. How
to
keep
Log
Processing
Apps
in
sync
with
CI?
Release
1
Release
2
Release
3
6

7. Our
Study
Dimensions
What
types
of
modifica*ons
happen
to
CI?
What
informa*on
is
conveyed
by
the
short-­‐lived
CI?
Quan4ty
Type
Content
How
does
CI
evolve
over
*me?
7

8. Case
Study
Setup
Data
Collec*on
Log
Abstrac*on
System
Deployment
*me=1,
Trying
to
launch,
TaskID=01A
*me=$t,
Trying
to
launch,
TaskID=$id
Enterprise
Applica*on
(EA)
8
Log
Events

9. Our
Study
Dimensions
What
types
of
modifica*ons
happen
to
CI?
What
informa*on
is
conveyed
by
the
short-­‐lived
CI?
Quan4ty
Type
Content
How
does
CI
evolve
over
*me?
9

10. CI
keeps
on
growing
over
4me
0
20
40
60
80
100
120
140
160
180
0.14.0
0.15.0
0.16.0
0.17.0
0.18.0
0.19.0
0.20.0
0.20.1
0.20.2
0.21.0
releases
#
execu*on
events
10

11. …even
when
system
size
decreases
#
K
SLOC
#
Execu*on
log
events
0.19.0
293
113
0.20.0
250
121
11

12. CI
is
impacted
by
re-­‐engineering
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
0.15.0
0.16.0
0.17.0
0.18.0
0.19.0
0.20.0
0.20.1
0.20.2
0.21.0
Unchanged
CI
Large
amounts
of
implementa4on
changes
12

13. How
does
CI
evolve
over
*me?
13
Growing
&
changing
Document
&
track
What
types
of
modifica*ons
happen
to
CI?
What
informa*on
is
conveyed
by
the
short-­‐lived
CI?
Quan4ty
Type
Content

14. Six
types
of
modifica4on
exist
Rephrasing
Redundant
informa4on
Adding
informa4on
Dele4ng
informa4on
Diverging
Merging
14

15. Six
types
of
modifica4on
exist
Rephrasing
Redundant
informa4on
Adding
informa4on
Dele4ng
informa4on
Diverging
Merging
Hadoop
mapred
Reduce
task
fetch
n
bytes
Hadoop
MapReduce
task
Reduce
fetch
n
bytes
15

16. Six
types
of
modifica4on
exist
Rephrasing
Redundant
informa4on
Adding
informa4on
Dele4ng
informa4on
Diverging
Merging
ShuffleRamManager
memory
limit
n
MaxSingleShuffleLimit
m
ShuffleRamManager
memory
limit
n
MaxSingleShuffleLimit
m
mergeThreshold
Q
16

17. Six
types
of
modifica4on
exist
Rephrasing
Redundant
informa4on
Adding
informa4on
Dele4ng
informa4on
Diverging
Merging
Adding
task
to
tasktracker
Adding
Map
Task
to
tasktracker
Adding
Reduce
Task
to
tasktracker
17

18. Six
types
of
modifica4on
exist
Rephrasing
Redundant
informa4on
Adding
informa4on
Dele4ng
informa4on
Diverging
Merging
Avoidable
18

19. Six
types
of
modifica4on
exist
Rephrasing
Redundant
informa4on
Adding
informa4on
Dele4ng
informa4on
Diverging
Merging
Recoverable
19

20. Six
types
of
modifica4on
exist
Rephrasing
Redundant
informa4on
Adding
informa4on
Dele4ng
informa4on
Diverging
Merging
Unavoidable
20

21. Most
modifica4ons
can
be
avoided
9.86%
61.97%
14.08%
7.04%
7.04%
2.82%
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
redundant
info
rephrasing
adding
info
dele4ng
info
diverging
merging
avoidable
recoverable
unavoidable
21

22. How
does
CI
evolve
over
*me?
22
Growing
&
changing
Document
&
track
What
types
of
modifica*ons
happen
to
CI?
What
informa*on
is
conveyed
by
the
short-­‐lived
CI?
Quan4ty
Type
Content
6
types
Are
mostly
avoidable

23. Short-­‐lived
CI
contains
implementa4on
details
Hadoop
saves
output
to
a
machine.
Hadoop
assigns
a
reduce
task
to
a
machine.
Map
task
updates
its
progress.
Hadoop
reads
from
a
local
file.
Hadoop
Abempt
saves
its
output
and
reports
to
the
task
tracker.
23
Node
name
Local
path
Using
ipc
Output
file
name

24. How
does
CI
evolve
over
*me?
24
Growing
&
changing
Document
&
track
What
types
of
modifica*ons
happen
to
CI?
What
informa*on
is
conveyed
by
the
short-­‐lived
CI?
Quan4ty
Type
Content
6
types
Are
mostly
avoidable
Implementa*on-­‐
level
details
Fragile
Maintenance
effort

25. 25