Simple measurements

Simple Measurements
Schalk W. Cronjé
ysb33r @ gmail.com
@ysb33r

Agile Cambridge 2011
© Schalk W. Cronjé

What is Agile?

Why are you not measuring
your process today?

How do you know when your
delivery was successful?


“What is Agile?” - Responses

●
“Continuous development done in small iterations”

●
“Adapting to rapid change, predictable periodic
delivery of usable software”

●
“Developers and testers work together from
beginning to end”

●
“You'd only get a cynical answer from me”


What is Agile Fundamentally?
●
Increased revenue
●
Reduction in costs
●
Faster response
times
●
Less rework

Delivering value to stakeholders
●
Customers
●
Users
●
Developers & Testers
●
Governments & Legislators
●
Societal Institutions

Opposing Forces?

Software Craftsmanship
Engineering

Accuracy Precision


Software Engineering

●
Implies measurement
●
Implies Kolb / Shewhart / Deming Cycles
●
Is not code craftmanship
●
SE receives bad press
– Perceived difficulty in measurement
– Lack of measurement in S/W dev


Accuracy

Accuracy is to precision
as
engineering is to mathematics


Accuracy

We need to be accurate enough
to get there, but not more accurate


Properties of a Kanban System


Taking an Economic View

●
It helps to quantify the effects of multiple
interacting variables
●
It helps us to understand that the customer is not
the only judge of value
●
By using an economic framework it can allow us to
maximise value, including
– cycle time
– product cost
– development expense
●
It helps to communicate with non-technical decision
makers

Background

3 Teams – 3 Locations – 18 months


David Anderson's Six Steps

●
Focus on quality
●
Reduce Work-in-Progress
●
Deliver often
●
Balance demand against throughput
●
Prioritise
●
Attack sources of variability to improve
predictability


Visualisation


Visualisation

●
Honesty!
●
Makes it transparent to everyone that is
going on
●
Visual cue for bottlenecks
●
Manual board and easy-to-use online system


Business Value Increments

●
Delivery is in business value increments
(BVIs)
●
Only “done” when the increment is
delivered end-to-end
●
No burning down of tasks ! (false economy)
●
Each increment must have at least one
metric of the value it will deliver


Batch Development
Feature 1 Dev Feature 1 QA

delivered
build


Batch Development
Total Time = devt1 .. devt4 + qat1 …qat4 + fixdelay + fixtime + retest time


delivered
build

defect trickle feed

Bug Feature 5+ QA
DB Retest fixes QA
delivered
build
Feature 5+ Dev


Time Factors
Building test
understanding infrastructure
specs
basic build
verification
delivered
build

time from raising defect until it is
available for testing
defect trickle feed time to
re-test

Bug Feature 5+ QA
DB Retest fixes QA
delivered
build
Feature 5+ Dev


"If you only quantify one thing,
quantify the cost of delay"

Don Reinertsen


Cost of Delay – Quality Issue

●
One small feature, not properly tested
●
Found post-release
●
Time wasting by users having to manual
work quantified => £35,000

TotalManualTimeSpent x AvgCostToCompanyPerUser +
TimeToRework x AvgCostToCompanyPerDev


Limit Work-in-Progress
Ready Spec Spec Dev Dev QA QA Released
Complete Complete Complete

●
Use a pull system
●
Managing queues are easier than managing timelines
●
Allows for better throughput utilisation
●
Identifies bottlenecks in the system
●
Measuring cycle time allows better prediction than
traditional estimation


Policies

●
Each stage has a policy
●
Describes generic activities required
●
“Definition of Done” for each stage


Feedback

●
Faster feedback makes learning faster and
more efficient
●
Faster feedback provides a sense of
control
●
Large batches leads to slower feedback


Measuring Cycle Time

●
Measure average time in system of each BVI
●
Less time is spent on estimating future delivery
times
●
Historic data automatically takes into account any
disruptions such as
– Operational issues
– Days off sick

AvgTimeInSystem x NoOfBVIs
Time = ---------------------------------
Concurrency

Cadences

●
Delivery is done at regular cadences.
●
What is ready is delivered, what is not, is
left out
●
No artificial time-boxing or break points as
in SCRUM-like approaches
●
Cycle-time leads to better predictability of
what can actually be delivered


Historic Data vs Estimation


Simple Flow Model
Specify Develop QA

Kanban
Board


Reducing Learning Time
●
Writing test specifications after the
development is costly in time
– Knowledge decay
– Leads to incomplete designs
●
Move the test specification up-front
before any development starts
– This is part of requirements discovery
– It broadens the perspective of the programmer
– Allows us to distinguish between automated
checks and human testing

Reducing Learning Time
Specify Develop QA

Test specification

+Time to specify -Time to re-work

+Time to develop -Time to re-test


Change the Flow Model
Specify Develop & Check Verify

●
Ensure the all automated checks are
included in the development stage
●
Free up a Verification stage for
exploratory testing and validating that
process has been fulfilled.


Change the Flow Model

+Time to develop -Time to re-work

+More testing people -Time to re-test


Real-world measurements

S1 S2 S3 S4
READY 4 6 5 1h
SPEC 1 0.5 6 0.5
SPEC/Completed 2 0.5 1 0.5
DEV + Check 2 1 13 4
DEV + Check / 3 6 4 18
Completed
VERIFY 3 7 8 1
VERIFY/Completed 2 5 6 18
Blockages 0.5 1h 2 1h
Total 17 26 43 42
Average cycle days per feature

What is wrong?

S1 S2 S3 S4
READY 4 6 5 1h
SPEC 1 0.5 6 0.5
DEV + Check / 3 6 4 18
Completed
VERIFY 3 7 8 1
Total 17 26 43 42

What is wrong?
Excessive long time between feature availability and testing. Is there enough
testing bandwidth? Or is this team just cheating?

S1 S2 S3 S4
READY 4 6 5 1h
SPEC 1 0.5 6 0.5
DEV + Check / 3 6 4 18
Completed
VERIFY 3 7 8 1
Total 17 26 43 42

Quantifying BVIs


Quantifying BVIs

●
Know what you want to achieve
●
Establish a scale for measurement
●
Decide how to measure
●
Make constraints explicit
●
Communicate


Quantify BVIs - Example

For all customer submissions that meets
the following criteria, 80% should be
handled by an automation system instead
of a human and resolution achieved within
30 minutes.

Criteria A
Criteria B
etc.


Planguage

●
Created by Tom Gilb
●
Provides a structured way of quantifying
requirements
●
Allows for reuse of specifications


Quantify BVI - Planguage

Name: Customer submissions resolved by automation systems

Scale: Percentage of submissions resolved by automation in
relation to all submissions

Meter: Query on database using the following statement:
SELECT … FROM … WHERE ...

Goal: 80% [Criteria A, Criteria B]


Applying the Economic Model

●
Compare to average human time to respond
and volume covered
●
Extrapolate to time savings or cost savings
●
This allows for two measures of success
– Technical
– Financial


Kanban fails when people
don’t want to face the truth.

- Hillel Glazer


It all fell apart ...

●
Restructure – 3 teams + 3 managers
●
2 managers rejected Kanban
– 1 manager wanted ScrumWorks
– 1 manager used Microsoft Project
●
1 manager was forced not to use Kanban
– Switched to iteration-based deliveries
●
All teams forced back to time-based
estimation


It fell apart even more ...

●
1 team decided on 4 wks dev + 4 wks
testing
– Quality was worse than before
– Cycle time was at least 8 calendar weeks per BVI
●
1 team did not deliver for over 5 months
– Finally delivered something the user did not want
●
None of teams have any measurements on
cycle time; cannot predict how long delivery
will take

What did the team members say?

●
“I have lost visualisation. I have no idea what is going
on”

●
“We are not using Kanban, because the company
forced us to SCRUM”

●
“If I was given a choice, my the team and I would
definitely go with Kanban”

●
“It is not the company 'standard' as such you need to
'sell' the process frequently. “

Lessons Learnt


Apply Lean Principles
to your Team

●
Make the system pull-based
●
Reduce the batch size
●
Limit the work-in-progress
●
One feature end-to-end
●
If the flow breaks, fix it immediately
●
Measure cycle time
●
Quantify requirements
●
Use Cost of Delay where applicable

Paired Teams

●
Pair up people with different skills to work
on same feature
– Feature teams are not a new concept
●
Could be perceived to have higher person
cost per feature
– Instead of distributing the person cost over
multiple queues, the cost is combined into a
single queue
– Can actually lead to reduced cycle time.

Simple measurements
are a foundation
of software engineering
and not for measuring humans


Simple measurements

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Simple measurements

Semelhante a Simple measurements (20)

Mais de Schalk Cronjé

Mais de Schalk Cronjé (20)

Último

Último (20)

Simple measurements