Velocity is one of the most commonly used - and abused - development metrics. Teams (and their stakeholders) often focus on “improving velocity” without either a proper consideration for root causes that impact velocity or a holistic view of a team’s outcomes.
Join Andy Cleff in a lively discussion that explores how we can remove perverse incentives and instead provide healthier ways for teams to gain meaningful insights into the outcomes of their experiments.
Metrics for Agile Teams Forget Velocity: 42 Other Things to Ponder
1. Metrics for Agile Teams
Forget Velocity, 42 Other Things to Consider!
Presented by
Andy Cleff
Sr. Agile Coach and Trainer
2. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Overview
12 rules
for measurement
5 Categories &
42+ metrics
How to choose?
Workshop
2
andycleff.com/metrics
3. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Poll: Who’s here?
What is your primary role? (pick one)
• Product Owner
• Dev-Team
• Scrum Master
• Agile Coach
• Management
• Other
3
POLL
4. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Poll Results: Who’s here?
4
POLL
5. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Andy.Cleff@
agilevelocity.com
AgileVelocity.com
Sr. Agile Coach
and Trainer
Andy Cleff
5
6. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
12 rules for measurement
1: Measure for a purpose
2: Shrink the unknown
3. Seek to improve
4: Delight all stakeholders
5: Distrust all numbers
6: Set imprecise targets
7: Own your metrics
8: Don’t connect metrics to rewards
9: Promote values and transparency
10: Visualize and humanize
11: Measure early and often
12: Try something else
change and innovation practices
MANAGEMENT 3.0
12 Rules for Measurement
When selecting metrics, ask:
Rule 1: Measure for a purpose
You must always understand why you are measuring.
The metric is not a goal in itself. Never forget that it’s
just a means to an end. It all starts with why.
Rule 2: Shrink the unknown
A metric is just a surrogate for what you really want to
know. Don’t jump to conclusions. Always try to reduce
the size of what is still unknown.
Rule 3. Seek to improve
Don’t only measure things that will make you look good.
There is plenty of data around, but you must focus on
what enables you to do better work.
Rule 4: Delight all stakeholders
Your work depends on others, and others depend on
you. Never optimize for just one stakeholder. Instead,
measure your work from multiple perspectives.
Rule 5: Distrust all numbers
attitude towards any reported numbers.
Rule 6: Set imprecise targets
When people have targets, they have an inclination to
focus on the targets instead of the real purpose. Avoid
this tendency by keeping your targets vague.
Rule 7: Own your metrics
Everyone is responsible for their own work, and metrics
help us improve that work. Therefore, everyone should
be responsible for their own metrics.
Rule 8: Don’t connect metrics to rewards
Rewards often kill intrinsic motivation and lead
to dysfunctional behaviors in organizations. Don’t
incentivize people to do work they should like doing.
Rule 9: Promote values and transparency
To prevent gaming, be transparent about values,
intentions, and the metrics everyone is using.
Rule 10: Visualize and humanize
Numbers tend to dehumanize everything. Replace digits
with colors and pictures, and keep the measurements
close to where the actual work is done.
Rule 11: Measure early and often
Most people don’t measure often enough. Measure
sooner and faster to prevent risks and problems from
growing too big for you to handle.
Rule 12: Try something else
It’s rarely a good idea to do the same things over and
over. The environment changes all the time. The same
should apply to how and what you measure.
Why “this metric?” – Why does it matter?
What insights might we gain from it?
What is expected to change? What is expected variability,
consistency – are we looking for trends or absolute values?
Working to improve one thing may temporarily reduce
promote knowledge sharing, collaboration with other
teams and trust with our sponsors?
Is this metric a leading or lagging indicator?
6
Shout out to Jurgen Appelo & Management 3.0
Big thanks to Ralph van Rosmalen for getting the ball rolling
7. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Rule 1- Measure
for a purpose
You must always understand
what you are measuring. The
metric is not a goal in itself.
Never forget that it’s just a
means to an end. It all starts
with why.
7
8. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Measure for outcomes, not outputs
Employee Engagement
Continuous Improvement
Predictability
Customer Satisfaction
Market Responsiveness
Productivity
Speed
Quality
Innovation
8
9. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Reasons why we do measure
To know revenues and drive resource & people allocation accordingly
To make decisions that are not based on gut feelings alone
Monitor alignment with mission / vision / goal
Observe quality of product / process
Judge customer happiness / employee satisfaction
9
10. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Reasons why we don’t measure
Measurements might be used as
weapons
Some metrics are not useful or actionable
Implementing measures would cost too
much time / effort
Some things might just be immeasurable
10
11. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Rule 2 - Shrink
the unknown
A metric is just a surrogate for
what you really want to know.
Don’t jump to conclusions.
Always try to reduce the size of
what is still unknown.
11
12. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Cynefin Framework
12
13. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Ordered
Domains
Cynefin Framework
13
OBVIOUS
CLEAR
14. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Here Things
Get Interesting…
Cynefin Framework
14
APORETIC
or
CONFUSED
15. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Rule 3 - Seek to
improve
Don’t only measure things that
will make you look and feel
good. There is plenty of data
around, but you must focus on
what enables you to do better
work.
15
16. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Actionable metrics
“A good metric changes the way you behave. This is by far
the most important criterion for a metric: what will you
do differently based on changes in the metric?”
Lean Analytics, Alistair Croll and Benjamin Yoskovitz
16
17. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Vanity metrics
“When we rely on vanity metrics, a funny thing happens.
When the numbers go up, I've personally witnessed
everyone in the company naturally attributing that rise to
whatever they were working on at the time. That's not too
bad, except for this correlate: when the numbers go
down, we invariably blame someone else”
Eric Ries
17
18. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Rule 4 - Delight all
stakeholders
Your work depends on others,
and others depend on you.
Never optimize for just one
stakeholder. Instead, measure
your work from multiple
perspectives.
18
19. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
It is impossible to please everyone, but you would like to
know who is pleased at certain moments and who is not.
19
What do your partners care about?
20. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Rule 5 - Distrust all
numbers
Observers usually influence their
own metrics, and they suffer
from all kinds of biases. Have a
healthy, skeptical attitude
towards any reported numbers.
20
21. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com 21
Cognitive biases
22. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Story
Hawthorne Works, Chicago, ca. 1925.
22
25. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Story
25
The Hawthorne, or Observer, Effect
26. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Rule 6 - Set
imprecise targets
When people have targets, they
have an inclination to focus on
the targets instead of the real
purpose. Avoid this tendency by
keeping your targets vague.
26
27. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
"When a measure becomes a target, it ceases to
be a good measure."
Goodhart's law
27
28. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Rule 7 - Own your
metrics
Everyone is responsible for their
own work, and metrics help us
improve that work. Therefore,
everyone should be responsible
for their own metrics.
28
29. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Important considerations
How many metrics should a team “own”?
Which ones?
How long should they use the ones selected?
29
30. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Anti-Patterns
People know/suspect they are being watched and judged (Hawthorn)
Striving for ever increasing values instead of striving for consistency
and stability (Goodhart)
30
31. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
More Anti-Patterns
Correlation is not necessarily causation (Milton Friedman’s Thermostat)
31
32. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
More Anti-Patterns
Comparing metrics
across teams that
are very different
32
33. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Poll: Scale of your ecosystem
Number of Teams (pick one)
• Less than 5
• Between 6 and 9
• More than 10
33
POLL
34. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Poll Results: Scale of your ecosystem
34
POLL
35. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Rule 8 - Don’t
connect metrics to
rewards
Rewards often kill intrinsic
motivation and lead to
dysfunctional behaviors in
organizations. Don’t incentivize
people to do work they should
like doing.
35
37. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Rule 9 - Promote
values and
transparency
Human beings are smart and
able to game any system. To
prevent gaming, be transparent
about values, intentions, and the
metrics everyone is using.
37
38. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Values, intention, purpose
Do we get paid a sustainable value for what we do?
Are we great at what we do in the eyes of our customers?
Do our employees / teammates love what we do and the
way we do it?
Will what we do make the world a better place for our
grandchildren?
38
39. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com 39
Values, intention, purpose
40. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Rule 10 - Visualize
and humanize
Numbers tend to dehumanize
everything. Replace digits with
colors and pictures, and keep
the measurements close to
where the actual work is done.
40
46. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Rule 11 - Measure
early and often
Most people don’t measure often
enough. Measure sooner and
faster to prevent risks and
problems from growing too big
for you to handle.
46
47. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com 47
“The only way to win is to learn faster than
anyone else”
Eric Ries
“What you want to do as a company is maximize
the number of experiments you can do per unit of
time.”
Jeff Bezos
On winning…
48. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Rule 12 - Try
something else
It’s rarely a good idea to do the
same things over and over. The
environment changes all the
time. The same should apply to
how and what you measure.
48
49. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Limited lifespan of all metrics
That which is measured will improve, at a cost.
When a measure becomes a target, it ceases to be a good
measure. (Goodhart)
Correlation is not causation, but it sure is a hint. (Friedman)
Use multiple viewpoints - technical as well as human - to
get a holistic perspective
49
50. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
12 rules for measurement
1: Measure for a purpose
2: Shrink the unknown
3. Seek to improve
4: Delight all stakeholders
5: Distrust all numbers
6: Set imprecise targets
7: Own your metrics
8: Don’t connect metrics to rewards
9: Promote values and transparency
10: Visualize and humanize
11: Measure early and often
12: Try something else
50
51. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Discussion / Q & A
Is anything confusing so
far?
What are you doing
already that's working?
What's not working for
you?
51
52. Optional
✦ When you return from the
break, ask yourself the
following question: “What is
the most important
concept I’ve learned so far?”
Break Over In:
Sponge Activity
53. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Toolbox of metrics
53
54. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com 54
Business Outcomes
55. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Poll: Your biggest “why”
What business outcome is the most important now? (pick one)
• Employee Engagement
• Continuous Improvement
• Customer Satisfaction
• Predictability
• Market Responsiveness
• Productivity
• Speed
• Quality
• Innovation
55
POLL
56. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Poll Results: Your biggest “why”
56
POLL
57. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Five categories of metrics
Process Health
Release
Product Development
Technical / Code
People / Teams
57
Shout out to:
Jason Tice @TheAgileFactor
58. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Process health metrics
Assess day-to-day delivery team activities and evaluate process changes:
• Cycle time
• Cumulative flow diagrams
• Control charts
• Number of experiments performed
• Number of improvements made to
process over time
• Flow efficiency
• “Concept to cash” delivery lead time
• Batch size
• Successful iteration completion
• Escaped defect resolution time
• Time thievery
58
59. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Shine a light on the time thieves
1. Too much work in progress (WIP)
2. External dependencies
3. Unplanned work
4. Conflicting priorities
5. Neglected work
59
60. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Cumulative Flow Diagram (CFD)
60
61. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Cumulative Flow Diagram (CFD)
61
kanbanize.com
Arrivals
Departures
WIP
62. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Cumulative Flow Diagram (CFD)
62
kanbanize.com
63. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Cumulative Flow Diagram (CFD)
63
Cycle Time
kanbanize.com
64. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Release metrics
Focus on identifying impediments to continuous delivery:
• Deployment frequency
• Release success rate
• Failed build frequency
• Mean time to restore (MTTR)
• Release time
• Cost per release
• Escaped defects
• Release net promoter score
• Release adoption / install rate
64
65. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Net promoter score (NPS)
65
surveymonkey.com
66. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Product development metrics
Help measure alignment of product features to user needs:
• Customer value delivered
• Risk burn down
• Value stream mapping
• Sales velocity
• Product net promoter score (NPS)
• User analytics / DAU / MAU
• Backlog health index
• Number of validated business-level
hypotheses developed
• Number of times a week you talk to
an actual customer
• Ratio of implemented to non-
implemented customer-driven
changes
• HEART
• AARRR
66
67. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
DAU/MAU
67
geckoboard.com
68. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
HEART Framework
68
Goal Signals Metrics
Happiness For users to feel like the site
is unique A satisfaction rating from a survey
Satisfaction
Perceived ease of use
Net promoter score
Engagement For users to keep
discovering more content
The amount to time people spend
viewing from analytics
Number of videos watched per user
per week
Number of uploads
Number of shares
Adoption For users to use the site or
app
The number of signup per days,
from analytics
Upgrades to latest version
New subscriptions
Purchased by new users
Retention For users to continue to use
the site or app
The number of return customers
from customer data
Number of active users
Renewal rate
Repeat purchase
Task Success For users to accomplish
their goal
The number of successful
completes from a study
Search result success
Time to upload
Profile creation completion
Google Ventures
69. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
AARRR (Pirate Metrics)
• Acquisition
• Activation
• Retention
• Referral
• Revenue
69
70. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Technical & code metrics
Help determine quality of implementation and architecture:
• Automated test coverage
• Number of tests written before coding
• Unit / regression test coverage
• Build time
• Defect density
• Code churn
• Code ownership
• Code complexity
• Coding standards adherence
• Crash rate / time to restore service
• Build breaks / change fail rate
• Technical drag
• Ratio of fixing work vs feature work
70
71. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Test automation dashboards
71
getzephyr.com
72. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
People & teams
Reveal issues that impact a team’s sustainable pace and level of engagement:
• Team health and well-being
• Team / manager / org eNPS
• % of time w/o interruptions (fire fights)
• Trust between leadership and team
• Learning log
• Overtime hours
• New employee setup time
• Team tenure / manager half-life
• Mean time to reorg
• Whole team contribution
• Transparency (access to data,
customers, shared learning)
• Employee silence vs voice
• Number of experiments / changes
72
73. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Why team health and well being matters
A strong constitution is necessary for teams to
overcome adversity, maintain resilience and
nurture a culture rich in collaboration and
innovation.
73
74. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Resilience
Dictionary: The capacity to adapt to stress or loss healthfully
• Positivity
• Focus
• Flexibility
• Organization
• Proactivity
74
Shout out to:
Daryl Conner
75. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Visualizing team health:
andycleff.com/teamhealth
Gallup Q12
Mood App + Health Monitor
TeamMood
eNPS
Comparative Agility Survey
Crisp Happiness Index
Team Barometer
Spotify Team Health Check
Happiness Wall / Window
Lean Agile Intelligence
Team Morale Metric
H.I.P. Survey
OfficeVibe
15Five
7Geese
75
76. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Team mood
76
teammood.com
Team members will receive a daily
email and/or slack prompt, asking for
their mood of the day. One click and
done!
77. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Team mood
77
teammood.com
78. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Measuring Team Culture
78
Forsgren PhD, Nicole. Accelerate . IT Revolution Press. Kindle Edition.
On my team….
79. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Measuring Trust and Ownership
79
80. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Measuring Trust and Ownership
80
81. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Measuring Trust and Ownership
81
82. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Measuring Trust and Ownership
82
83. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Discussion / Q & A
Is anything confusing so
far?
What are you doing
already that's working?
What's not working for
you?
83
84. Optional
✦ When you return from the
break, think about what
you’ve learned so far and
make a list of the names of
colleagues with whom you
could share some or all of
the information you’ve
picked up so far.
Break Over In:
Sponge Activity
85. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Care and usage As George Box said, “All models
are wrong, but some are useful.”
85
86. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Integral lenses
I / Personal:
This is about “Me” –
my psychological,
inner world; how I see
things, what I
experience, my
mindset.
86
We / Relational:
A shared vision and
interpersonal
dynamics among
peers; how we
interact, what we
value together, the
team micro-culture.
Its / Environmental:
Things external to the
team systems,
including other value
streams, handoffs,
rules, corporate
culture, and policies.
It / Practices:
Refers to the team
structure, methods,
metrics, processes,
decision making
patterns, leadership
and team practices.
87. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Who a metrics model is for
There are two groups of
stakeholders:
• The individuals & teams
(I + We)
• The people & systems
supporting the teams
(It + Its)
I
Personal
Subjective
IT
Practices
Objective
WE
Relational
Intersubjective
ITS
Environment
Interobjective
Individual / Singular
Collective / Multiples
Interior /
Invisible
Exterior /
Visible
87
88. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
From the team perspective
• Builds up awareness
• Reveals working and what’s not
• Expands perspectives
• Provides a balanced view
88
89. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
From the supporting role perspective
• Provides high level summary
• Reveals patterns across
teams
• Shows opportunities to
change the system
89
90. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
What this is not
• A competition
• A comparison
• A weapon
90
91. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
What this is not
• An incentivisation model
91
92. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
What this is not
• Done
92
93. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
How to choose?
93
94. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Review the options….
Why “this metric?” – Why does it
matter? To whom does it matter?
What insights might we gain from it?
What is expected to change? Are we
looking for variability, consistency,
trends, values, direction?
94
95. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Review the options….
How might a metric be gamed,
misused (or abused)?
What are some trade offs (collateral
damage) as well as the costs of
getting “better data”?
95
96. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Review the options….
How often would we like to “take a
data point”?
How long will we run the
experiment?
How when we know when we’re
“done” with this metric?
96
97. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Review the options….
How will we make this measurement
transparent?
Is this metric a leading or lagging
indicator?
97
98. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
If you’re really in a hurry to get started…
1 = Choose from People & Teams
2 = People & Teams + track
average age of items in process
3 = People & Teams + any two from
Little’s Law (CT = WIP / TH) and then
you’ll get one bonus for free!
5 = People & Teams + Little’s Law +
Customer NPS + Time Since Last
Release
98
99. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Summary so far…
• Use predictability to delight
• Abandon old concepts
• Measure many things (just not too many)
• Learn rapidly
• More geek joy
99
100. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Discussion / Q & A
Is anything confusing so
far?
What are you doing
already that's working?
What's not working for
you?
100
101. Optional
✦ When you’ve returned from
the break, and before you sit
down, take a couple of really
deep breaths. Maybe stretch
your arms overhead too.
✦ Then add a comment about
what you like best about
the course so far.
Break Over In:
Sponge Activity Print out worksheets from:
andycleff.com/metrics
102. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Putting Theory
into Practice
102
andycleff.com/metrics
103. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
1. Explore what’s going well along with your main pain
points in the context of your desired business outcome
2. What items are a priority to experiment with?
3. Brainstorm what metrics you might try. Come up with
some good ones, and some bad ones too…
4. Work thru the “when selecting metrics” questions…
Workshop: Overview
103
andycleff.com/metrics
104. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Workshop: Paint your canvas…
1.
2. What items are a priority to experiment with?
3. Brainstorm what metrics you might try. Come up with
some good ones, and some bad ones too…
4. Work thru the “when selecting metrics” questions…
104
1. Explore what’s going well along with your main pain
points in the context of your desired business outcome
105. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Example: What’s going well / main pain points
105
+ Team does lunch and learns
! Too much “must fix now”
! Sprints over capacity
! No ability to A/B test ideas
! Infrequent releases
106. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Workshop: Paint your canvas…
1.
2. What items are a priority to experiment with?
3. Brainstorm what metrics you might try. Come up with
some good ones, and some bad ones too…
4. Work thru the “when selecting metrics” questions…
106
1. Explore what’s going well along with your main pain
points in the context of your desired business outcome
107. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Workshop: Paint your canvas…
1. Explore what’s going well along with your main pain
points in the context of your desired business outcome
2. What items are a priority to experiment with?
3. Brainstorm what metrics you might try. Come up with
some good ones, and some bad ones too…
4. Work thru the “when selecting metrics” questions…
107
108. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com 108
Example: Priority to experiment with
0% slack time 20% slack time 6 wks 2x
109. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Workshop: Paint your canvas…
1. Explore what’s going well along with your main pain
points in the context of your desired business outcome
2. What items are a priority to experiment with?
3. Brainstorm what metrics you might try. Come up with
some good ones, and some bad ones too…
4. Work thru the “when selecting metrics” questions…
109
110. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Workshop: Paint your canvas…
1. Explore what’s going well along with your main pain
points in the context of your desired business outcome
2. What items are a priority to experiment with?
3. Brainstorm what metrics you might try. Come up with
some good ones, and some bad ones too…
4. Work thru the “when selecting metrics” questions…
110
111. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com 111
Example: What metrics you might try
Customer NPS
DAU/MAU
eNPS / Team Mood
Story points per sprint
DAU (alone)
112. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Workshop: Paint your canvas…
1. Explore what’s going well along with your main pain
points in the context of your desired business outcome
2. What items are a priority to experiment with?
3. Brainstorm what metrics you might try. Come up with
some good ones, and some bad ones too…
4. Work thru the “when selecting metrics” questions…
112
113. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Workshop: Paint your canvas…
1. Explore what’s going well along with your main pain
points in the context of your desired business outcome
2. What items are a priority to experiment with?
3. Brainstorm what metrics you might try. Come up with
some good ones, and some bad ones too…
4. Work thru the “when selecting metrics” questions…
113
114. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Workshop: Paint your canvas…
Assumptions and corresponding tests:
• We believe that [persona type] has [trouble/need/desire]
doing [problem/opportunity].
• We will know we have succeeded when [qualitative
outcome] and [quantitative outcome]. This will improve
[kpi / measurable signal].
114
115. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com 115
Example: Assumptions and corresponding tests
We believe that our active users
has a desire for more frequent
releases that include innovative
social sharing features.
We know we have succeeded when:
• DAU/MAU reaches 50% within 3
months…
• Dev-Team mood on average is above
7/10…
• Velocity decreases 20%…
116. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Workshop: Paint your canvas…
Assumptions and corresponding tests:
• We believe that [persona type] has [trouble/need/desire]
doing [problem/opportunity].
• We will know we have succeeded when [qualitative
outcome] and [quantitative outcome]. This will improve
[kpi / measurable signal].
116
117. Sharing
•What are the things you’ll measure?
•How will you learn rapidly?
•Will this move you towards predictability?
•How will you visualize things?
•How will your information help you make better decisions?
117Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
118. If you get stuck…
•How much? As in, how many “widgets”
are you producing each week?
•How fast? How long does it take to create that item
from the second the conversation starts to when it’s finished?
•How good? What’s the quality?
•How predictable? If you tell a customer to expect an item within a certain
number of days, how frequently do you meet that deadline?
•How happy? Is your team happy and is your culture healthy?
118Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
119. If you get stuck…
119Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Business
Outcome Example Measures
Employee
Engagement
• eNPS
• Bad turnover
• Employee referrals
• Offer acceptance
• Feedback surveys
• Team learning logs
Continuous
Improvement
• Value stream flow efficiency
• Reduction in recurring impediments
• Cumulative flow
120. 120Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Business
Outcome Example Measures
Innovation • Market Share
• Number of validated business-level hypotheses developed
and tested
• Slack time
Customer
Satisfaction
• Number of times a week team members talk to an actual
customer
• NPS
• Retention (DAU/MAU)
• Referrals
121. 121Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Business
Outcome Example Measures
Market
Responsiveness
• AARRR (Acquisition, Activation, Retention, Referral,
Revenue)
• HEART, (Happiness, engagement, adoption, retention, task
success)
Predictability • % complete of Sprint plan
• Velocity variability
• Say-Do ratio
• Unplanned work items
Speed • Cycle time
• Lead time
• Deployment frequency
• Meantime to restore (MTTR)
122. 122Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Business
Outcome Example Measures
Quality • Defects in production
• Change fail percentage
• Automated test coverage
• Ratio of fixing work vs feature work
Productivity • Value delivered
• ROI
• KPIs
• Thieves of time
123. Metrics for Agile Teams • Andy Cleff • AgileVelocity.Com
Let’s keep the conversation going…
If you need to work on your foundational practices...
If you need to re-focus...
If you feel stuck mid-transformation...
If you need ongoing support during your transformation…
Please contact us, we offer:
• Private Agile Training
• Agile Assessments
• Agility Tune-ups
• Agile Transformation Services
123