A look at some of the methodologies that have shaped the direction of agile software development. We take a look at Lean Software Development (and the Toyota Production System), the Theory of Constraints and Systems Thinking.
1. THE FOUNDATIONS OF AGILE
Lean Software Development
Theory of Constraints
Systems Thinking
2. What is agile?
How would we describe agile?
• A set of patterns, practices and tools
• Part technical practices
• Part management theory
• Part cult?
3. The prevailing mindset...
• All about the division of labour
• Adam Smith‟s pin factory
• Frederick Winslow Taylor‟s Scientific Management
• Interchangeable part and interchangeable labour
• Little respect for workers, assumes they are lazy and want to
do bare minimum
• Grim for the workers
5. The Toyota Production System
• Vision is to have just in time assembly
• Aim to remove
• Overburden (muri)
• pushing people or machines beyond their limits
• inconsistency (mura)
• build quality into the system to avoid inconsistency
• waste (muda)
• 7 wastes ; over production, waiting, transportation, processing, inventories /
WIP, movement, defects
• Allowed TPS to reduce lead time and cost whilst improving
quality
6. TPS Principles
• Continuous improvement - Kaizen
• Respect for people - build trust, stimulate professional
and personal growth
• Long Term philosophy, even at the expense of short term
capacity
• Right Process, right results - "pull" work to avoid
overproduction, stop the line mentality, visualise problems
• Develop people and partners - develop people who follow
your companies mentality
• Drive organisational learning through solving root
problems
8. Theory Of Constraints
“a chain is no stronger than its weakest link”
• Introduced by Eli Goldratt in 1984 book „The Goal‟
• The goal of a systems is always limited by at least one
constraint, and increasing the flow through the constraint
will increase throughput
9. 5 focusing steps
1. Identify the constraint
2. Decide how to exploit the constraint
3. Subordinate everything else to the constrain -
align to the constraint
4. Elevate the constraint - increase the constraints
capacity
5. If the constraint has been broken identify the
new constraint and start again
10. The key points from TOC
• Local optimisation may not (most probably will not) lead to
optimisation of the whole.
• It is not necessary to have maximum efficiency from non
constraints.
Some things it shares with TPS
• Continuous improvements through employee
empowerment
• Cultural transformation
• Global rather than local goals
12. Lean
• When the consultants got hold of TPS in the late 80 / 90s
there was no stopping it
• Many tried to replicate, but championing the tools and
processes without mindset and principles often failed
• Lean was adapted to many different sectors
• Lean manufacturing, Lean operations, Lean supply chain, Lean
product development etc etc.
13. Lean Software Development
• Established by Mary and Tom Poppendieck in 2003
Has 7 principles
1) Eliminate waste - identify it, remove anything that does
not add customer value
2) Build quality in - not test it in later, don't create the
defects in the first place
14. Principles (continued)
3) Create knowledge - base it on facts from feedback
4) Defer commitment - make irreversible decisions at the
last responsible moment, make decisions reversible or at
least provide a framework to support changes
5) Deliver fast - get working software to customers
15. Principles (continued)
6) Respect people - Excellent leaders, expert workforce,
self organisation
7) Optimize the whole - avoid the temptation to sub
optimize. optimizing a subsystem will almost certainly sub
optimise the whole
17. What is a System
• Most things are a system or part of a system
• A set of things interconnected in such a way that produce
their own pattern of behaviour over time.
• Has a function or purpose
• Can be very simple or very complicated
• Systems are powered by feedback. Behaviour emerges
as a result of the inputs
• Interactions are not linear - "Cause and effect are not
closely related in time and space" - Peter Senge
18. William Edwards Deming
• Born at the height of Taylorism, Deming studied statistical
process techniques
• He took his statistical process control methods to Japan
after WW2 to high acclaim and success
19. Deming
• The aim is to improve quality in order to reduce expense
and increase productivity.
• As quality increases costs will fall
• But, focusing on reducing costs will cause costs to rise
• Created the PDCA cycle
20. Deming‟s system of profound knowledge
1. Appreciation of system - understand the system as a
whole
2. Knowledge of variation - understand the variation in the
system
3. Theory of knowledge - how do we explain and share
knowledge?
4. Knowledge of psychology - how does human nature
affect the system
21. Peter Senge
• Took earlier systems thinking ideas to the create the concept of
a "learning organization“
At its heart are 5 disciplines
1. Personal mastery - a spirit of continual learning and working
towards a personal vision
2. Mental models understand how ingrained assumptions and
generalizations influence how we see the world and act
3. Build a shared vision - build genuine commitment rather than
compliance
4. Team learning through dialogue - suspend assumptions and
learn together
5. Systems thinking - The fifth discipline that integrated the
other 4
22. Peter Senge‟s Laws
• Today's problems come from yesterday's "solutions."
• The harder you push, the harder the system pushes back.
• Behaviour grows better before it grows worse.
• The easy way out usually leads back in.
• The cure can be worse than the disease.
• Faster is slower.
• Cause and effect are not closely related in time and space.
• Small changes can produce big results...but the areas of
highest leverage are often the least obvious.
• You can have your cake and eat it too …but not all at once.
• Dividing an elephant in half does not produce two small
elephants.
• There is no blame.
23. Final thought from Systems Thinking
• It is not possible to mandate performance outside of the
capability of the system
• A bad system will defeat a good people every time
• Need to stop seeing the world as linear relationships and
think more about underlying interactions
• Understand the demand on the systems
• You are not your job role, you are not a column on a
scrum board
24. Further reading
• Deming - Out of the crisis (maybe don't start with this
one)
• Mary and Tom Poppendieck - Lean software development
• Donella Meadows - Thinking in systems
• Peter Senge - The Fifth Discipline
• Eli Goldratt - The Goal
• Tons more stuff out there
Notas do Editor
Disclaimer: I am not and expert in any of the fields. Feel free to shout up at any point if there are any questions.Agile can sometimes feel like a cult, especially if it is blindly followed without questioning why.We talk a lot about the technical practices but often neglect to understand the management theory behind agile.This is not just for 'management'. It is relevant to all people involved in an agile organisation.- It is about setting the context for the things we do
Before:Software development is a very young field, so it isn't surprising that the prevailing ideas come from the field of manufacturing. What might be surprising is that some of the underlying ideas are fairly old.It is also worth remembering that although some of the ideas we are going to talk about have their roots in manufacturing, these ideas are not about manufacturing. They are equally applicable in knowledge work.This is not a complete history, I am only focusing on a few key people and ideas. There are many more.Prime example being Ford’s Model T production lineSee also Theory X and Theory Y
Before:The early 20th century starts to see investigations into alternative ways of working. A couple of names will appear again and again; TaiichiOhno and William Edwards Deming, who will see more of later...
JIT => Producing cars at the rate of customer demandTPS is the source of most of the agile related Japanese works you will hear banded around
Can you see some agile themes emerging here?TPS allowed Toyota to enjoy a lot of success, so this has to make a massive impact in other organisations, right? Nope. Largely ignored in Japan until external conditions (oil crisis) ended the boom of the 60’s. Ignore by the rest of the world for a lot longer.
Whilst Toyota was thinking about waste, Eli Goldratt was thinking about Constraints
EliGoldratt wrote the book whilst working at a software company making software for mapping manufacturing processesPeople like the book more than the softwareHe was sacked. Founded a TOC institute instead
1) Identify the constraint*How do we identify the constraint?*- Often work piles up in front of it. Scrum boards make it visible2) Decide how to exploit the constraint*What in agile allows us make this kind of decision* Retrospectives give teams chance to discuss3) Subordinate everything else to the constrain - align to the constraint, protect with WIP limits*maybe protect with WIP limits4) Elevate the constraint - increase the constraints capacity*How?* Change roles within teams - generalisation over specification, swarming etc5) If the constraint has been broken identify the new constraint and start again.
Although TOC has some salient points, it never mapped easily to software development. Many of the ideas have been further evolved into the concepts around kanban and continuous flow.It is no accident that one of the key people people in the introduction of kanban into SD (David Anderson) was also a driving force behind the adoption of the ideas of TOC into SD
Blindly copying the tool and practices is unlikely to lead to good resultsEventually we arrive at Lean Software Development - based on the principles of Lean
Principles relate closely to the original Lean principles1) Eliminate waste - identify it, remove anything that does not add customer value*What are the main forms of waste in SD?* Partially done work, requirements churn, un-required features*How do we avoid churn and un needed features?* Specify stories just in time2) Build quality in - not test it in later, don't create the defects in the first place.*How?* TDD, Acceptance testing, CI
3) Create knowledge - base it on facts from feedback- feedback comes from building software, not just specifying it, release early (CD?), generate fast feedback (tests)4) Defer commitment - make irreversible decisions at the last responsible moment, make decisions reversible or at least provide a framework to support changes- planning is not the same as making a commitment.*Flexible architectures, acceptance tests* e.g. NCMP5) Deliver fast - get working software to customers*Showcase demo, short iterations, small stories delivering value*
6) Respect people - Excellent leaders, expert workforce, self organisation*retrospectives, dojos, organisational learning*7) Optimize the whole - avoid the temptation to sub optimize. optimizing a subsystem will almost certainly sub optimise the whole*What situations apply to this?It is no good having an optimized development team if upstream and downstream teams are not geared up to work at the right cadence.*A good time to look at Systems Thinking
Systems thinking is a massive topic, so will focus on a key few people and ideas.
A set of things interconnected in such a way that produce their own pattern of behaviour over time.Has a function or purpose*but the purpose of a system is what it does, not what it says it does*It is possible to see the purpose of a system in different ways**What is the purpose of a supermarket? Supply products to customers? To generate a profit for owner? to provide employment?*What is the purpose of software development?Systems are powered by feedback. Behaviour emerges as a result of the inputs. *Feedback could be reinforcing e.g. bank interest, market share, birth rates, or balancing e.g. death rate,*Interactions are not linear - "Cause and effect are not closely related in time and space" - Peter SengePeople are good at seeing linear relationships, but we often get caught out by linear thinking, causation versus correlationWhat are the examples in SD when cause and effect are not closely related in time and space?- Technical debt- problems with analysis cause issues and rework later in the process- Team 'safety' (how safe do the team feel from blame and recrimination) effects estimation. Unsafe teams increase estimates and game metrics as a defense mechanism. Incentives do not always work as expected
*It is hard to separate out Deming's work into a separate field. It certainly influence TPS and TOC, and widely influences agile today*
This is part of a wider generalization that focusing to improve/reduce X will usually have the opposite effect.What does this look like in SD?- increasing output by skipping unit testing or dropping technical practices
1) Appreciation of system - understand the system as a whole*So it SD this is the entire value chain*2) Knowledge of variation - understand the variation in the system2 kindsCommon cause variation - The variation inherent in the system. The system noise.*What common cause variation exists in SD? - story and task estimation versus the time taken, velocityGenerally within knowable limits based on historic performancespecial cause - a signal of a new or emergent phenomenon. It is non historical and cannot be predicted. It is usually evidence of some underlying change in the system* illness, new team members, departmental restructuring**SD, and knowledge work in general is a high variability industry.*3) Theory of knowledge - how do we explain and share knowledge?4) Knowledge of psychology - how does human nature affect the system.*Understand incentives and coercion. Cognitive biases, Dunning Kruger effect, impostor syndrome**Culture**With the results he achieved in Japan he must have made a massive impact in the States. right? Nope. Largely ignore almost until his death in 1993.
1) Personal mastery - a spirit of continual learning and working towards a personal vision.2) Mental models understand how ingrained assumptions and generalizations influence how we see the world and act3) Build a shared vision - build genuine commitment rather than compliance4) Team learning through dialogue - suspend assumptions and learn together* Does this ever happen?* What does agile have to help with this? Retrospective - prime directive?5) Systems thinking - The fifth discipline that integrated the other 4*Again, culture is a key part*
*He came up with some snappy laws as well**There is loads more - Systems Thinking deserves its own Dojo*
It is not possible to mandate performance outside of the capability of the system- Need to change the underlying culture and conditions and the performance will emergeA bad system will defeat a good people every time*Deming 95%* discussNeed to stop seeing the world as linear relationships and think more about underlying interactionsUnderstand the demand on the systems*John Seddon's concept of demand - 2 types failure demand and value demand**What is the failure demand on SD?* Rework, requirements churn - like the waste in Lean SD*You are not your job role, you are not a column on a scrum board
There are lots more ideas an people who have shaped agile