cPrime's latest Agile Meetup discussion will center around methods for how to monitor and validate the performance of agile. Many firms that have been doing agile can not determine how or if it has had an impact on the company. Agile expert, Jeff Howey will discuss ways to evaluate agile performance. Join our webinar to learn how to identify the benefits of agile and uncover the differences between companies that exhibit "agile-like" behavior and highly functioning teams.

1. Agile
Performance
Metrics
Presenter:
Jeff
Howey
–
Agile
Coach,
CSM
4100
E.
Third
Ave,
Suite
205,
Foster
City,
CA
94404
|
650-­‐931-­‐1651
|
www.cprime.com
The
leader
in
training
and
consulGng
for
project
management
and
agile
development

2. ©
2012,
cPrime.
All
Rights
Reserved

3. ! If
you
have
a
quesGon,
please
type
it
in
the
“QuesGons”
Chat
Window.
Our
team
will
collect
quesGons
and
include
responses
to
many
of
them
when
we
post
the
material
online.
! We
will
post
a
copy
of
the
presentaGon,
a
recording
of
the
webinar
and
follow-­‐up
informaGon
on
www.cprime.com
! We
will
send
email
with
links
! Feel
free
to
share!
©
2012,
cPrime.
All
Rights
Reserved

4. ! What
is
the
current
adopGon
of
Agile
processes
in
your
organizaGon?
! PracGcing
Agile
processes
on
most/all
projects
! PiloGng
Agile
on
select
projects
! Considering
Agile
for
future
projects
! Not
considering
Agile
in
the
foreseeable
future
©
2012,
cPrime.
All
Rights
Reserved

5. And
What
Metrics
Will
Demonstrate
Impact?
©
2012,
cPrime.
All
Rights
Reserved

6. ! Don’t
we
value
working
so/ware
over
comprehensive
documenta6on?
! Why
do
I
need
tools
and
processes
if
our
focus
is
on
individuals
and
interac6ons?
! Is
management
measuring
my
performance
based
on
how
many
story
points
I
complete
when
my
goal
is
to
increase
customer
sa6sfac6on?
! Do
my
stakeholders
really
care
about
numbers
as
long
as
I
am
responsive
to
my
customer’s
requests?
©
2012,
cPrime.
All
Rights
Reserved
-­‐
Excerpts
from
the
Agile
Manifesto

7. ! The
OrganizaGon
as
a
whole
needs
to
understand
how
and
why
Agile
is
making
a
difference
in
project
delivery
! The
Agile
Team
needs
insight
into
areas
they
can
improve
! Agile
sGll
values
processes,
tools
and
documentaGon!
! No
more
“fake
it,
‘Gl
you
make
it”
jusGficaGons
! You
need
to
“move
it,
‘Gl
you
prove
it”
! And
keep
proving
it
©
2012,
cPrime.
All
Rights
Reserved

8. 2010 Project Success
21% Fail 37%
Success
42% Standish Chaos Reports
Challenge Project Success
60
% 50,000 Projects
50
40
Challenged
•
late
(100%
median)
30
•
over
budget
(50%
median)
20
•
lacking
funcGonality
•
low
quality
10
0
1994
1996
1998
2000
2004
2006
2009
2010
%
Success
16
27
26
28
29
35
32
37
%
Challenged
53
33
46
49
53
46
44
42
%
Failed
31
40
28
23
18
19
24
21
Data from 50,000 projects
Copyright © by The Standish Group International, Inc.
©
2012,
cPrime.
All
Rights
Reserved

9. ! How
would
you
rate
your
team’s
history
of
delivering
projects?
! More
successes
than
failures
! More
failures
than
successes
! About
the
same
as
Standish
reports
! I’d
rather
not
say
©
2012,
cPrime.
All
Rights
Reserved

10. Feature
Usage
by
Frequency
How
much
did
this
cost???
16%
Always
45%
64%
Ohen
19%
13%
SomeGmes
Rarely
7%
Never
64% of product features are rarely or never used
2006
–
The
Standish
Group
©
2012,
cPrime.
All
Rights
Reserved

11. • The
average
Fortune
500
sohware
project
costs
$3.2
million
and
takes
1,290
(resource)
months
to
complete.
Yet
only
37%
of
respondents
answered
"yes"
when
asked
if
their
projects
met
users'
needs
•
2008
–
Voke,
Inc.
©
2012,
cPrime.
All
Rights
Reserved

12. ! “In
2002,
agile
projects
made
up
less
than
2%
of
overall
projects
and
less
than
5%
of
new
applicaGon
development
projects.
Today,
agile
projects
account
for
almost
9%
of
all
projects
and
29%
of
new
applicaGon
development
projects...
The
increase
in
project
success
rates
can
directly
@e
back
to
projects
resolved
through
the
agile
process.”
2011
Chaos
Manifesto
Copyright © by The Standish Group International, Inc
©
2012,
cPrime.
All
Rights
Reserved

13. ! Predictability
! Delivery
Date
! Delivery
Content
! Reliability
! Of
EsGmates
! Of
Requirements
! Quality
! IniGal
Quality
! Cost
of
Defect
ResoluGon
! Customer
SaGsfacGon
©
2012,
cPrime.
All
Rights
Reserved

14. ! Understand
History
! What
is
your
personal
best/worst/average
in
each
measure?
! Set
a
Baseline
Target
! At
least
aim
to
improve
1-­‐3
measures
in
the
next
3-­‐6
months
without
hurGng
any
others
metrics
! Avoid
serng
aim
to
improve
everything
immediately
! Set
Future
Targets
! ConGnue
improvement
of
key
measures
! Target
to
improve
some
others
©
2012,
cPrime.
All
Rights
Reserved

15. ©
2012,
cPrime.
All
Rights
Reserved

16. ! How
ohen
do
you
make
major
releases?
! Monthly
or
more
frequently
! Quarterly
! Every
6-­‐12
months
! Once
a
year
or
less
frequently
©
2012,
cPrime.
All
Rights
Reserved

17. ! Somewhat
related
to
Reliability
of
EsGmates
but
disGnctly
different
! Target
dates
should
be
met
without
“heroics”
of
the
team
! Baseline
from
History
! What
%
of
projects
deliver
within
a
range
+/-­‐
the
iniGal
date?
! By
how
much
are
projects
delivered
late
when
they
are
not
within
a
given
range?
! Set
Goals!
! Somewhere
between
“best”
and
“average”
history
! Definitely
beter
than
“worst”
©
2012,
cPrime.
All
Rights
Reserved

18. Project
ID
Delivery
Plan
Date
Actual
Delivery
Delta
from
Plan
Total
Days
in
Date
Project
1
3/31/2009
4/30/2009
+30
days
180
2
12/31/2009
12/31/2009
0
days
180
3
3/31/2010
7/31/2010
+210
days
480
4
3/31/2010
5/31/2010
+60
days
60
5
6/30/2011
7/15/2011
+15
days
180
6
6/30/2011
12/15/2011
+165
days
480
7
1/31/2012
4/30/2012
(est)
+90
days
90*
Average
Variance
of
Release
Date
from
Plan
! Mean:
35%
late
! Median:
33%
late
What
is
your
reality?
©
2012,
cPrime.
All
Rights
Reserved

19. ! First,
understand
WHY
target
dates
have
been
missed
in
the
past
! What
can
Agile
address?
Uncertainty
or
Changing
Scope
in
Requirements,
unrealisGc
dates
for
full
scope
compleGon,
etc.
! What
can
Agile
not
solve?
Corporate
culture,
ritual-­‐
intensive
release
processes,
etc.
! Brainstorm
soluGon
opGons
! PrioriGze
opGons
to
pursue
! Inspect,
Adapt
and
maintain
Transparency
while
implemenGng
soluGon
alternaGves
©
2012,
cPrime.
All
Rights
Reserved

20. ! Define
a
Target
%
of
Releases
that
make
the
date
! Adjust
scope
based
on
priority
to
deliver
funcGonality
that
is
good
enough
ON
the
planned
date
! Define
a
beta-­‐type
release
strategy
where
feasible
! Especially
in
organizaGons
with
long/complex
release
procedures
for
producGon
availability
! IdenGfy
where
can
you
release
a
beta
version
and
receive
valuable
feedback
©
2012,
cPrime.
All
Rights
Reserved

21. ©
2012,
cPrime.
All
Rights
Reserved

22. ! Comparison
of
! IniGal
Requirements
vs.
Delivered
Requirements:
What
%
of
funcGonality
was
removed
from
scope?
! Were
undelivered
requirements
rescheduled
for
later
release?
! Were
features
given
up
out
of
frustraGon?
! Not
always
bad
to
deliver
less
than
planned
! Were
features
not
needed?
! Were
they
simplified
-­‐
but
sGll
“good
enough”
for
users?
! If
it
was
actually
a
“savings”
or
diverted
focus
to
higher
prioriGes,
it
could
be
a
benefit
©
2012,
cPrime.
All
Rights
Reserved

23. ! How
many
implemented
stories
were
rejected
by
the
Product
Owner
at
a
Sprint
Review?
! What
were
the
Drivers?
! incomplete
requirements?
(Product
Owner
has
opportuniGes
to
improve)
! incomplete
implementaGon?
(Agile
Team
has
opportuniGes
to
improve)
©
2012,
cPrime.
All
Rights
Reserved

24. ! Agile
Teams
-­‐
Set
goals
around
the
number
of
stories
that
are
rejected
from
the
Product
Owner
once
implemented
! Is
the
right
number
for
you
80%?
95%?
©
2012,
cPrime.
All
Rights
Reserved

25. ©
2012,
cPrime.
All
Rights
Reserved

26. ! Look
at
Story
Points
first
! Aher
implementaGon,
perhaps
during
RetrospecGves,
replay
Planning
Poker®
! “Knowing
what
you
know
now,
what
would
be
the
esGmate
for
this
story”
! Measure
all
stories
(big
and
small)
to
keep
fair
track
items
! Compare
and
Average
variances
! Break
down
by
Small/Medium/Large
! Is
the
team
beter
at
esGmaGng
in
one
area
vs.
others?
! Adjust
the
esGmaGng
process
as
needed
©
2012,
cPrime.
All
Rights
Reserved

27. ! As
the
team
refines
their
EsGmaGng
Process,
conGnue
to
review
the
Product
Backlog
and
re-­‐
esGmate
upcoming
User
&
Technical
Stories
! Review
the
“ Top
1/3”
of
the
Product
Backlog
if
Gme
allows
for
the
first
Release
or
first
5-­‐7
Sprints
of
a
new
project
or
new
team
composiGon
! Reset
and
baseline
Story
Points,
conGnue
to
track
progress
toward
beter
esGmaGon
! Keep
this
grooming
process
short!
! PerfecGon
is
expensive
and
unatainable
! Set
limits
(1
hour
per
Sprint
during
Story
Review
MeeGngs,
perhaps?)
©
2012,
cPrime.
All
Rights
Reserved

28. ! At
the
Sprint
Backlog
(hours)
level
vs.
the
Product
Backlog
level
(Story
Points)
! IdenGfy
any
recurring
procedures
that
may
be
candidates
for
project
standards
and
assign
hours
esGmates
to
include
in
future
Sprints
! Compare
Actual:EsGmated
hours
for
tasks
and
understand
variances
! They
are
expected,
so
idenGfy
paterns
and
don’t
dwell
for
more
than
5
minutes
on
specific
reasons
! For
instance,
“programming
stored
procedures
are
always
underesGmated”
or
“Joe
is
new
to
Java
so
takes
25%
more
Gme
than
Sally
does
at
coding
right
now”
! Adjust
esGmaGng
process
to
account
for
paterns
©
2012,
cPrime.
All
Rights
Reserved

29. ©
2012,
cPrime.
All
Rights
Reserved

30. ! What
%
of
Stories
are
prioriGzed
for
change
to
the
iniGal
implementaGon
in
the
following
3
Sprints?
Why?
! Focus
criGcal
atenGon
on
Stories
that
are
prioriGzed
for
change
in
the
Sprint
immediately
following
iniGal
implementaGon
! Product
Owners
–
This
really
is
one
of
your
biggest
measures
! Does
this
indicate
you
need
to
do
more
research
or
refinement
of
Stories
prior
to
Sprint
Planning?
©
2012,
cPrime.
All
Rights
Reserved

31. ©
2012,
cPrime.
All
Rights
Reserved

32. ! Just
kidding
–
0
defects
is
expensive
and
likely
indicates
shortcomings
in
the
tesGng
process!
! IdenGfy
historical
number
of
defects
idenGfied
post-­‐release
by
priority
(P1,
P2,
etc.)
! Set
goals
by
Release
to
drasGcally
reduce
the
number
of
defects
idenGfied
post-­‐release
! Set
goals
and
standards
to
minimize
or
eliminate
high-­‐priority
defects
to
consider
a
Sprint
“done”
! For
example,
the
Agile
Team
sets
a
standard
that
they
cannot
exit
a
Sprint
with
outstanding
P1’s
©
2012,
cPrime.
All
Rights
Reserved

33. ©
2012,
cPrime.
All
Rights
Reserved

34. ! Track
at
an
Aggregate
Level
and
begin
to
IdenGfy
where
defects
are
being
resolved
or
(ideally)
prevented
! Agile
Teams
&
Product
Owners
are
all
on
the
hook!
©
2012,
cPrime.
All
Rights
Reserved

35. ©
2012,
cPrime.
All
Rights
Reserved

36. ! UlGmately
this
is
the
only
metric
that
used
to
compare
Agile
Teams
! Story
Points
for
Blue
Team
<>
Story
Points
for
Green
Team
! Velocity
for
Blue
Team
<>
Velocity
for
Green
Team
! Defect
Rate
for
Blue
Team
<>
Defect
Rate
for
Green
Team
! IdenGfy
your
customers
–
Internal
&
External
! Track
what
is
important
to
Your
Customer!
! Time
to
Market
! Length
of
Visit
per
page
! Click-­‐through
Rate
! Ideally
track
and
monitor
per
Release
©
2012,
cPrime.
All
Rights
Reserved

37. ! Use
a
4-­‐point
Scale
vs.
5
or
10
! Forces
a
“3”
or
“4”
RaGng
and
avoids
“Average”
! 4
keeps
it
a
simple
comparison
vs.
10
! Words
like
“Awful,
Could
Be
Beter,
It
Works,
Great”
are
more
powerful
than
“Poor,
Fair,
Good,
Excellent”
! Be
sure
to
capture
comments!
! Allow
input
on
every
quesGon,
not
just
a
generic
feedback
text
entry
secGon
! AcGvely
Solicit!
“ Tell
us
what
could
be
beter”
is
more
powerful
than
“Comments”
©
2012,
cPrime.
All
Rights
Reserved

38. ! Record
feedback
and
scores
for
future
reference
! During
RetrospecGve
and
Story
Review
meeGngs,
review
the
feedback
and
keep
it
fresh
! Adapt
funcGonality,
schedule
or
whatever
elements
are
important
to
the
Customer
and
given
as
feedback
! Incorporate
“What
could
we
do
beter?”
feedback
into
features
or
processes
©
2012,
cPrime.
All
Rights
Reserved

39. ©
2012,
cPrime.
All
Rights
Reserved

40. ! Defects
per
developer
! Legacy
performance
evaluaGon
criteria
! NOT
valid
with
Agile:
self-­‐organizing
teams
should
allow
individuals
to
learn
new
things
–
the
“test-­‐n-­‐
learn”
process
anGcipates
defects
! Lines
of
Code/Story
Points
per
developer
or
team
! My
5-­‐point
Story
is
NOT
your
5-­‐point
Story
(this
is
true
at
the
individual
and
team
level)
! What
other
metrics
do
you
think
are
missing?
! Compare
to
the
Agile
Manifesto
and
Principles
to
validate
their
use
©
2012,
cPrime.
All
Rights
Reserved

41. ! Once
goals
are
set
for
any
set
of
metrics,
these
steps
are
CRITICAL
! ConGnue
to
measure
against
goals
! Review
as
an
Agile
Team
! Share
with
Product
Owners,
Management,
Customers
! ConGnuously
Refine
goals
! Do
not
tackle
everything
at
once!
! PrioriGze
what
is
most
important
to
the
Agile
Team
and
the
Customer
to
address
first
! Take
on
new
challenges
when
goals
are
met
©
2012,
cPrime.
All
Rights
Reserved

42. ©
2012,
cPrime.
All
Rights
Reserved

43. ! How
much
money
is
6ed
up
in
projects
that
are
approved
but
delayed
or
“on
deck”
! Can
we
prove
that
releasing
more
frequently
based
on
prioriGzed
features
helps
the
organizaGon
more
wisely
invest
in
Products
&
Projects?
! This
is
tricky
to
measure
and
even
more
tricky
to
track,
but
it
is
CRITICAL
©
2012,
cPrime.
All
Rights
Reserved

44. ! Do
you
have
projects
that
are
approved,
but
are
overdue,
delayed
or
sirng
"on
deck"
without
resources
to
work
them?
! Yes,
and
maybe
Agile
can
help
us
manage
our
investments
beter
! Yes,
but
I
don’t
think
Agile
would
help
! No,
we
only
allocate
budget
for
projects
when
resources
are
ready
to
go
! None
of
the
above
quite
fit
©
2012,
cPrime.
All
Rights
Reserved

45. ! As
organizaGons
implement
Agile,
it
is
criGcal
to
track
the
$
invested
in
projects
and
the
speed
at
which
the
investments
are
realized
through
Product
Releases
! Supports
buy-­‐in
and
evangelizaGon
of
Agile
benefits
where
Agile
makes
sense
©
2012,
cPrime.
All
Rights
Reserved

46. ! How
much
money
has
been
spent
on
the
64%
of
features
that
are
rarely
or
never
used?
Can
Agile
put
that
to
beter
use
in
the
future?
! Do
you
track
usage
of
features?
! Through
automaGon
in
the
applicaGon
! Via
User
Survey
! Product
Owners
need
this
informaGon
to
prioriGze
work.
IT/Project
Management
needs
this
to
understand
impact
on
Por•olio
©
2012,
cPrime.
All
Rights
Reserved

47. Create
Your
Own,
but
Here
are
Some
Ideas
©
2012,
cPrime.
All
Rights
Reserved

48. Metric
DescripIon
Target
*
Sprint
Predictability
FracGon
of
Sprints
in
which
all
planned
Stories
>
90%
are
completed
Deliverable
Number
of
Stories
rejected
by
Product
Owner
<
1
RejecGon
Rate
at
end
of
Sprint
Requirement
Number
of
Stories
rejected
in
Sprint
Planning
<
1
RejecGon
Rate
MeeGng
EsGmaGon
Accuracy
of
Sprint
Backlog
esGmate
±20%
of
hours
in
Reliability
80%
of
Sprints
Tracking
Reliability
No.
of
Tasks
with
incorrect
Status
per
day
<
5%
Defect
Impact
Effort
/
Sprint
for
P0,
P1
ProducGon
bugs
(i.e.,
<
20%
more
effort
devoted
to
value
delivery)
Quality
P0,
P1
ProducGon
bugs,
per
Release
3X
reducGon
Cost
of
Defects
Cost
to
correct
ProducGon
bugs,
per
Release
3X
reducGon
*
Example
Goal:
“ The
Team
aims
to
achieve
these
aOer
3
Releases”
©
2012,
cPrime.
All
Rights
Reserved

49. Metric
DescripIon
Time
to
Market
Time
from
request
to
implementaGon
for
highest-­‐value
features
Rate
of
Value
Delivery
Value
delivered
per
quarter
/
release
Adaptability
How
well
/
frequently
do
we
change
course
successfully,
when
needed?
Predictability
Extent
to
which
we
deliver
planned
features
on
schedule
(excepGng
change
requests)
Customer
SaGsfacGon
How
well
are
we
meeGng
customer
needs?
Customer
InteracGon
How
well
do
we
involve
customers
in
defining
needs
and
gerng
feedback?
Internal
CollaboraGon
How
well
are
we
meeGng
internal
needs
of
various
departments?
(Support,
Sales,
etc.)
These
are
more
tricky
to
track
but
very
useful
©
2012,
cPrime.
All
Rights
Reserved

50. ! How
did
we
do?
! I
learned
quite
a
few
things
that
I
can
use
! A
good
reminder
of
the
basics
and
a
few
new
ideas
! I’m
not
sure
how
I
would
apply
most
of
what
I
heard
! Waste
of
Gme
©
2012,
cPrime.
All
Rights
Reserved