Just collecting, storing and analyzing data is not enough. In order to benefit from it, you have to overcome organizational and human inertia and establish automated processes that use insights gained from your data.
This presentation has been presented at http://dataconomy.com/28-august-2014-big-data-berlin/
15. HOW DATA-DRIVEN DECISIONS SHOULD WORK
COMPUTER
COLLECTS
COMPUTER
STORES
HUMAN
ANALYZES
HUMAN
PREDICTS
HUMAN
DECIDES
16. HOW DATA-DRIVEN DECISIONS REALLY WORK
COMPUTER
COLLECTS
COMPUTER
STORES
HUMAN
ANALYZES
COMMUNICATION
BREAKDOWN
HUMAN
DECIDES
17. COMMUNICATION
BREAKDOWN
Communication Breakdown, It's always the same,
I'm having a nervous breakdown, Drive me insane!
— LED ZEPPELIN
• Drill-down analysis … misunderstood or distorted
• Metrics dashboards … contradictory and confusing
• Monthly reports … ignored after two iterations
• In-house analyst teams … overworked and powerless
19. HOW DECISIONS REALLY SHOULD WORK
COMPUTER
COLLECTS
COMPUTER
STORES
COMPUTER
ANALYZES
COMPUTER
PREDICTS
COMPUTER
DECIDES
20. 99.9% of all business decisions can be automated
21. HOW DECISIONS ARE MADE
Human Decisions
Business Rules
No Decision, Decision-by-default
22. BUSINESS RULES = PROGRAMMING
• Business rules are like programs
– written by non-programmers
• Business rules can be contradictory, incomplete,
and complex beyond comprehension
• Business rules have no built-in feedback
mechanism: “It is the rule, because it is the rule”
23. HUMAN DECISION
MAKING
• • System 1: Fast, automatic,
frequent, emotional, stereotypic,
subconscious
• • System 2: Slow, effortful,
infrequent, logical, calculating,
conscious
DANIEL KAHNEMANN, THINKING FAST
AND SLOW
24. HOW TO DECIDE FAST
FREQUENT DECISION MAKING MEANS FAST DECISION MAKING,
MEANS USING HEURISTICS OR COGNITIVE BIASES
Anchoring effect
IKEA effect
Over-justification effect
Bandwagon effect
Confirmation bias
Substitution
Availability heuristic Texas Sharpshooter Fallacy
Gambler’s fallacy
Illusory correlation
Rhyme as reason effect
Hindsight bias
Zero-risk bias
Framing effect
Sunk cost fallacy
Overconfidence
Outcome bias
Inattentional Blindness
Benjamin Franklin effect
Anecdotal evidence
Negativity bias
Loss aversion
Backfire effect
25. HOW COMPUTERS DECIDE FAST
MACHINE LEARNING OFFERS AN ALTERNATIVE TO HUMAN
COGNITIVE BIASES AND CAN BE MADE FAST THROUGH BIG DATA
K-Means Clustering
Markov Chain Monte Carlo
Support Vector Machines Naive Bayes Affinity Propagation
Decision Trees
Nearest Neighbors
Least Angle Regression
Logistic Regression
Spectral clustering
Restricted Bolzmann Machines
27. HOW PREDICTIVE APPLICATIONS WORK
COLLECT &
STORE
ANALYZE
CORRELATIONS
BUILD
DECISION
MODEL
DECIDE &
TEST OPTIMIZE
28. WHY TEST?
“Correlation doesn’t imply causation, but it does waggle its
eyebrows suggestively and gesture furtively while mouthing ‘look
over there’”
–RANDALL MUNROE
31. THE GROUND BEEF DILEMMA
Yesterday Today Tomorrow Next Delivery Next Day
In Stock Demand
32. THE GROUND BEEF DILEMMA
• Order too much and you will have to throw meat
away when it goes bad. You lose money and cows
die in vain
• Order too little and you won’t serve all your
potential customers. You lose money and
customers stay hungry.