Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1FbZwcD. Big Design Upfront was considered so evil in the early days of Agile that it acquired its own acronym. But absent the careful thinking of good designers, our systems turn out to be mediocre collections of stuff that customers asked for or the data showed was desirable. It’s time we relearned that great products start with asking the right questions. Filmed at qconnewyork.com. Mary Poppendieck has been in the Information Technology industry for over thirty years. She has managed software development, supply chain management, manufacturing operations, and new product development. Mary is a popular writer and speaker, and coauthor of the book Lean Software Development, which was awarded the Software Development Productivity Award in 2004.

1. l e a nsoftware development
www.poppendieck.comMary Poppendieckmary@poppendieck.commary@poppendieck.com
Design vs. Data
Enemies or Friends?

2. InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/design-data

3. Presented at QCon New York
www.qconnewyork.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide

4. August 152 Copyright©2015 Poppendieck.LLC
How do we avoid this?

5. August 153 Copyright©2015 Poppendieck.LLC
Notre Dame de Paris 1163-1345
1850

6. August 154 Copyright©2015 Poppendieck.LLC
Cologne Cathedral 1248-1473 & 1842-1880
1824

7. August 155 Copyright©2015 Poppendieck.LLC
Poitiers Cathedral (~1160 – 1290)Chartres Cathedral (~1134-1220)
1160
1510

8. August 15 Copyright©2015 Poppendieck.LLC
Strong Centers
The VoidContrast
Local Symmetries
Echoes
(Patterns)
Good Shape
Simplicity
Boundaries
Positive Space
Alternating Repetition
(Recursion)
Not-Separateness
(Connectedness)
Levels of Scale
Theory of Centers* – Fifteen Properties of “Wholeness”
Roughness
*Christopher Alexander
Deep Interlock
(Ambiguity)
Gradients

9. l e a n
Properties of Wholeness for Software*
Levels of Scale
Strong Centers
Boundaries
Symmetry
Recursion
Patterns
Space
Shape
Simplicity
Connectedness
August 15 Copyright©2015 Poppendieck.LLC7
(Left Out):
 Ambiguity
 Roughness
 Contrast
 Gradient
 Void
Hypothesis:
Learning through ongoing
experimentation is not an
excuse for sloppy system
design. On the contrary:
Strong systems grow from
a design vision that helps
maintain “Properties of
Wholeness” while learning
through careful analysis
and rigorous experiments.
*With credit to Christopher Alexander

10. l e a n
A Design Vision to Make
Data a Strong Center
Understand Data
and how to use it.
 Data is central
 A picture is worth a
thousand data points
 Appreciate statistics
 Analyze / Experiment
– know the difference
 Everyone is on
the same team
Simplify the Job
of Data Scientists.
 Data pipelines must
be wide and fast
 Develop API’s for
learning and control
 Experiments need
design and structure
 Use an architecture
that supports learning
August 158 Copyright©2015 Poppendieck.LLC
See, Think, Gain
Amazing Insights
 Know how to use the
best tools and models
 Be explicit about
assumptions
 Share the search for
patterns and outliers
 Get inside the data
 Test insights rigorously
Space
Simplicity
Levels of Scale
Shape
Boundaries
Connectedness
Patterns
Symmetry
Recursion

11. l e a n
How we Use Data
Monitor Control
Simulate Predict
August 159 Copyright©2015 Poppendieck.LLC

12. l e a n
Monitor
August 15 Copyright©2015 Poppendieck.LLC10
Something
went wrong
and the
system is
down. Lots of
people are
frantically
looking under
every rock,
and finding
dozens of
things that
are not right.
Pinpoint the moment in time when everything went wrong.
What went wrong in 1984?
Mickey Dickerson

13. l e a n
Control
August 15 Copyright©2015 Poppendieck.LLC11
Controllers regulate
temperature,
position, speed,
pressure, flow,
weight, force,
thickness,
chemical
composition,
and practically
every other variable
that can be measured.
P = Size of the error
D = Rate at which
error is changing
I = Residual error that
accumulates over time

14. l e a n
Simulate
For many embedded systems
lack of hardware, complexity of
hardware, or inconvenience of
setup limits the ability to use
Continuous Integration (CI).
Simulation makes it possible
to use standard PCs and
servers to run code destined
for even deeply embedded
target systems, making an
effective CI possible that is
more flexible and cheaper
than relying on hardware alone.
August 15 Copyright©2015 Poppendieck.LLC12
Credit for this page: Jakob Engblom, Wind River, Kista, Sweden

15. l e a n
Predict
August 15 Copyright©2015 Poppendieck.LLC13
1. Insights:
Extracted from data.
2. Predictions:
What will happen,
when, what will be
affected, and what
factors influences it.
3. Optimizations:
The most valuable
use of computational
analytics is knowing
“what to do next”.
The new arrival – the Data Scientist

16. l e a n
Exploratory Data Analysis (EDA)
An approach/philosophy for data analysis that employs a
variety of techniques (mostly graphical) to maximize
insight into a data set
1. Uncover underlying structure
2. Extract important variables
3. Detect outliers and anomalies
4. Test underlying assumptions
5. Develop frugal models
6. Determine optimal factor settings
August 15 Copyright©2015 Poppendieck.LLC14
All the Data
Sample
Try fitting
various curves
Variables that help create
models that fit sample data
Test promising
variables on all data
Variables that
work on all data
Potential
Models

17. l e a n
A systematic approach to problem-solving that applies principles
and techniques at the data collection stage that ensure the generation
of valid, defensible, and supportable conclusions.
1. Comparative
Has a change in a single factor
changed the whole process?
2. Screening/Characterizing
Form a ranked list of factors
that affect the process.
3. Modeling
Create a mathematical model
of the process.
4. Optimizing
Determine the optimal settings
for the process factors.
Design of Experiments (DOE)
August 15 Copyright©2015 Poppendieck.LLC15
Screening
Experiments
Mixture
Experiments
Response
Surface
Analysis
Fractional
Factorial
Experiments
Full
Factorial
Experiments
Evolutionary
Operations
(EVOP)

18. l e a n
Design for Data
Basic Elements:
Fast Pipelines
Data Wrangling
Data Analytics
Data Visualization
Designed Experiments
Machine Learning
Adaptable Business Systems
and Processes (ready and able
to use insights and services)
August 15 Copyright©2015 Poppendieck.LLC16
Credit: Ravi Kalakota, Liquid Analytics

19. l e a n
Design the System (not just the code)
August 1517 Copyright©2015 Poppendieck.LLC
Design like a Master Mason
Rather than a Stonecutter

20. l e a nsoftware development
www.poppendieck.comMary Poppendieckmary@poppendieck.commary@poppendieck.com
Thank You!
More Information: www.poppendieck.com

21. Watch the video with slide synchronization on
InfoQ.com!
http://www.infoq.com/presentations/design-
data