A presentation to Interspire, Sydney (July, 2011), looking at patterns of software growth and change, and strategies for managing complexity in large legacy applications.
2. Why does software change?
Adding a feature
Growth
Behavior is modified
Fixing a bug
Repair
Improving structure
Refactoring
Behavior stays the
same
Improving performance
Optimization
3. The balance of value
!
Creating features and fixing bugs adds value
directly to the product
!
We can measure the success of these activities by
monitoring conversions and sales
!
Performance optimization is empirical
!
We get hard data on whether we did it right or not
!
Structural change is difficult to quantify
!
Impacts on features, bugs, and performance
!
Highly subjective
4. Why structural change matters
!
Software growth is not constant
!
Bursts of activity; release and QA cycles
!
We can’t anticipate all future requirements
!
We want change with minimum cost and effort
!
Requires code to be supple and adaptable
!
We want predictable results from change
!
Requires effects of code changes to be localized
!
Requires loosely coupled, easily testable objects
5. Measuring Programming
!
Creative activity: no statistical feedback to tell us
whether we are reaching our goal in code
!
Little or no correlation between time spent coding
and quality of the result
!
We can mine data from code syntax; version
control history; bug trackers
!
But the same metrics have different meanings,
depending on the particular team and codebase
6. Looking at code on a larger scale
!
Methods per class 70
!
God class 60
!
Lines per method 50
!
Complex method 40
Revisions per code file
Metric
!
30
!
Churn 20
!
Bug fixes per code file 10
!
Instability
0
7. Statistical Similarities
!
All large code bases seem to share common
statistical features
!
The shapes of many software metrics conform to
power law or log-normal distributions
!
AKA: 80/20 rule, Pareto Principle, ‘long tail effect’
8. The shape of software
!
These statistical discoveries give us a new way of
looking at widely held ideas about software
engineering:
!
The ‘Big Ball of Mud’ is the wrong metaphor — even
messy legacy systems have a definite shape
!
The ‘Lego Hypothesis’ is invalid — object oriented
software is not constructed by connecting reusable,
modular components
9. What we say versus what we do
!
Traditionally, we describe effects like overly long
methods and complex classes as ‘code smells’
!
But these effects seem to be common to every
large code base in existence
!
As software engineers, we have to account for this
in our methodology
10. Behavioral Economics
!
Psychological explanation for why code tends
towards certain shapes
!
Stronger incentives for getting something working
quickly than sacrificing time and mental effort to
create new abstractions
!
Easier to reason about code changes in one place
!
Naming things is one of the most difficult problems
in software development
11. Actor Networks
!
Sociological explanation for why code tends
towards certain shapes
!
Conway’s Law: structure of systems tends to
mirror that of the organization building the system
!
The boundary of an application is an entirely social
construction
!
Rationale for coding decisions is lost when
developers leave the team or new requirements
overwhelm an existing system
12. Accidental Complexity
!
A by-product of the technology used to solve the
problem
!
Cost of change increases with the volume of code
!
Code that is difficult to change
!
Unavoidable
!
Indicator of code not evolving to fit changing
requirements
!
Indicator of a mismatch between chosen abstractions
and the shape of the problem domain
13. Granularity
!
Choice of programming language matters
!
Translating a high-level concept into actual code
!
Coarse grained APIs are inflexible and require heavy
customization and workarounds
!
Fine grained APIs are hard to use and require large
amounts of wrapping code and sequential calls
!
Architectural style:
!
Convention or Configuration
!
Library or Framework
14. How Buildings Learn
!
Layers of a building grouped according to different
rates of change:
!
STUFF
!
SPACE-PLAN
!
SERVICES
!
SKIN
!
STRUCTURE
!
SITE
15. How Applications Learn
!
Layered architecture
!
Lower levels wrap infrastructure that changes less
frequently
!
Upper levels wrap business logic that changes
constantly
!
Domain driven design
!
Minimize accidental complexity through a rigorous
approach to language
!
Look for implicit models hidden in existing code and
flesh them out
16. Ubiquitous Language
!
A living document of all the terms, concepts, and
relationships that drive product features
!
Bridging product level thinking and technical
architecture
!
Models behaviour from a software point of view
!
Encapsulates essential complexity
!
Changes and evolves in sync with the product
17. Relentless Refactoring
!
Refine the model; improve the language
!
Discover new organizing concepts
!
Focus on the relationships between components
!
Each object has a single responsibility
!
Small pieces, loosely joined
!
Break apart entanglements
!
Extract new classes and methods from existing code
!
Extract new applications from existing code
18. Component Boundaries
Application Logic
!
Dependencies are
transitive
Domain Model !
Lower level packages
should never depend on
higher level packages
Data Store
19. Evolving Architecture Patterns
!
Strangler App
!
Introduce new architectural abstractions that wrap the
existing system like a creeping vine that eventually
overtakes the host tree
!
Parallel Rewrite
!
New architecture develops side by side with the old
system within a single app
!
Extract App
!
New applications and service layers are spawned off
the original system