Working with Data Workshop

Working with data
define - disassemble - evaluate - act
workshop [comp_name] [date]

jankyri.com | mail@jankyri.com
You know nothing John Snow?
source:
John Snow - the original uploader was Rsabbatini at English Wikipedia CC BY 4.0, via Wikimedia Commons,
https://commons.wikimedia.org/wiki/File:John_Snow.jpg
Water pump by Justinc, CC BY 2.0, https://en.wikipedia.org/wiki/File:John_Snow_memorial_and_pub.jpg
- Physician
- London
- ~1810 - 1860
- Has a memorial plus a pub
named after him!

Cholera epidemic
London
1854
Cause: unknown
extremely deadly:
600 people in ~15 days
Theory 1:
“foul” air
Solution:
washing
Theory 2:
contaminated water
Solution:
Use clean water
Cholera: two competing hypotheses

Cholera: two competing hypotheses
source:
https://en.wikipedia.org/wiki/File:Snow-cholera-map-1.jpg
Theory 1:
“foul” air
Solution:
washing
Theory 2:
germs in contaminated water
Solution:
Use clean water

What is this?
● a workshop
● 60% theory, 40% practice
● learn to
● define
● disassemble
○ data
● evaluate
○ facts
○ context
● act on
○ insights

Why?
Goal:
● data literacy
○ equip you with a mental model for data, facts and insights
○ get a feeling for
● assumptions
● forecasts
● act on insights
○ understand your work better
○ find ways to get clarity and improve
Nobody is born a master analyst. All of you can learn this.

1. Define
1. define

What is data?
1. define

What is data?
Data is the new oil!
Clive Humby
source:
http://ana.blogs.com/maestros/2006/11/data_is_the_new.html
1. define

What is data?
Data is just like crude.
It’s valuable, but if unrefined it cannot really be used.
It has to be changed into gas, plastic, chemicals, etc to create a valuable entity that
drives profitable activity; so must data be broken down, analyzed for it to have
value.
First, we must understand that data is not insight.
Clive Humby
1. define

What is data?
data
qualitative quantitative
labels
● good
● average
● bad
● green
● cold
● fluid
continuous
● 900.54 €
● 1.5 s
● 33.56 %
discreet
● 147 signups
● 80 clicks
● 5 conversions
1. define

What is data?
insight
explicit:
fact
data
implicit:
context
action
1. define

2. Disassemble
2. disassemble

2. Disassemble
2. disassemble
source:
https://en.wikipedia.org/wiki/File:Hieroglyphs_from_the_tomb_of_Seti_I.jpg

2. Disassemble
Average customer rating:
“good”
2. disassemble

2. Disassemble
Revenue: 160,465.89 €
Clicks: 8,562
Lines of code: 1,000
2. disassemble

2. Context matters
Revenue: 160,465.89 €
Clicks: 8,562
Lines of code: 1,000Average customer rating: “good”
qualitative quantitative
2. disassemble

2. Context matters
Source: https://www.sheffield.ac.uk/faculty/social-sciences/news/self-employed-happier-more-engaged-at-work-
1.770210
19 March 2018
Self-employed people happier and more engaged at
work, study finds
Self-employed people are happier and more engaged in their work than those in any other
profession, according to a new study of 5,000 workers.
The self-employed workers who took part in the research worked in a range of industries,
including management consultancy, financial services, retail, education, insurance and real estate.
2. disassemble

2. Known vs. unknown entities
Things we
know
don’t know
know
don’t know
know
don’t know
facts that may be wrong
and should be checked
against data.
questions we can
answer by reporting,
which we should
baseline and automate.
intuition, which we
should quantify and
teach to improve
effectiveness and
efficiency.
exploration, which is
where unfair
advantages and
epiphanies live.
Source:
Donald Rumsfeld press briefing 2002, adapted from: https://en.wikipedia.org/wiki/Johari_window
Croll, Alistair and Yoskovitz, Benjamin, Lean Analytics: Use Data to Build a Better Startup Faster, O’Reilly, 2013
2. disassemble

3. Evaluate
3. evaluate

3. Evaluation Introduction
Things we
know
don’t know
know
don’t know
know
don’t know
facts
questions
intuition
exploration
easy
medium
hard
ease to
evaluate
medium
large
low
NA
impact if
wrong
medium
3. evaluate

3. Introduction: Wrong intuition and bias is human
"[Apple's iPhone] is the most expensive phone in the world and it doesn't appeal
to business customers because it doesn't have a keyboard …"
Steve Ballmer, 2007
"There's just not that many videos I want to watch."
Steve Chen, CTO and co-founder of YouTube expressing concerns about his company’s long term
viability, 2005
"There is no reason anyone would want a computer
in their home."
Ken Olson, Digital Equipment Corp., 1977
3. evaluate

3. Introduction: Wrong intuition and bias is human
But:
Predictions are very important
subjective → objective reality
Why are predictions tricky?
3. evaluate

3. Human Psychology: Kahneman
System 1:
“Thinking fast”
gut reaction
System 2:
“Thinking slow”
analytical
thinking
% of activity
3. evaluate

3. Human Psychology in Evaluation: WYSIATI
WYSIATI = what you see is all there is
Law of small numbers
generalizing small
# of observations to
population
Confirmation bias
focus on limited evidence
confirming your mental
story, fail to seek contrary
facts
Overconfidence
ignoring what you
don’t know, focus on
what easily comes to
mind
Hindsight bias
focus on past
outcomes, not on
quality of process
Over-optimism
Plan with best-case
scenarios instead of
weighing costs, risks and
potential blocks
3. evaluate
Source: https://icons8.com/icon/set/brain/doodle

3. Overcoming WYSIATI with science
“Doing data analysis without
explicitly
defining your
problem or goal is like
heading out on a road trip
without having decided
on a destination.”
Source:
Head First Data Analysis, Milton, Michael, O’Reilly, 2009
3.1 Have a hypothesis
X X
3. evaluate

● Your world view determines your analytical view
○ Thinking slow → Thinking fast
● Predictions make your assumptions and uncertainty explicit
● groups
○ can balance uncertainty
○ increase accuracy
3.2 Make your predictions
3. evaluate

● compare facts
○ raw data (outliers)
○ interval comparison (week, month, year)
■ beware of small # of data points!
● Seek contrary evidence,
○ talk to people → adapt the outside view
3.3 Test and falsify your predictions
3. evaluate

3. Evaluation: best practices
scientific method
● come up with a hypothesis
● make a prediction
● do
○ Ask colleagues for prediction
○ Conduct research to falsify your prediction
○ Conduct an experiment to test the prediction
3. evaluate

3. Game time: Flash forecasting
Forecast intervals with a confidence level
1) Get pen & paper
2) Write down confidence interval, lower bound, upper bound, draw distribution
Example:
80% confidence level = you are right 4 / 5 guesses
If you don't know anything about the subject at all and think
1 %,
50 %
and 99 %
are equally likely to be the true answer, your bounds should be something like 10-90 %.
3. evaluate
Source:
https://www.aforeseeablefuture.com

Ready?
3. evaluate

Game time
What is the share of working persons in Germany in 2018?
Example:
~ 80 million inhabitants (December 2017)
80 % confidence level:
8 - 72 million
3. evaluate

Game time
Age in Germany: nominal and relative distribution
Source:
http://www.gbe-bund.de, Indikator 1 der ECHI shortlist: Bevölkerung nach Geschlecht und Alter
Source:
https://service.destatis.de/bevoelkerungspyramide/#!y=2017
3. evaluate

Game time
Source:
https://www.aforeseeablefuture.com
How old was Martin Luther King Jr. when he was
assassinated?
3. evaluate

Game time
How many percent of the world population have access to the internet?
3. evaluate

Game time
More questions at aforeseeablefuture.com
or
build your company’s question set!
3. evaluate

Be conscious about System 1 thinking and actively engage System 2!
● Are your ranges
○ wide? → signals your assumed distribution
○ small? → possibly you are overconfident
● What is the distance of the actual value to your bounds?
○ does your mental image of the distribution of data reflect the real world?
● What is your accuracy over time?
○ are you prone to a certain bias?
○ is there an area you can learn more about to increase your accuracy?
Source:
https://hbr.org/2016/05/superforecasting-how-to-upgrade-your-companys-judgment
3. evaluate

Be conscious about System 1 thinking and actively engage System 2!
● individuals’ forecasting ability improves already after 1 hour
● use teams boosts accuracy
● track prediction performance and provide rapid feedback
○ use past data for training
○ possibly monthly challenges
→ develop a “sanity compass” for facts
3. evaluate

3. Forecasting: winning as a group
Process: Avoid anchors!
1) diverging phase, in which the issue, assumptions, and approaches to
finding an answer are explored from multiple angles
2) evaluating phase, which includes time for productive disagreement
3) converging phase, when the team settles on a prediction
Trust among members of a team is key for good outcomes (speak your mind)!
3. evaluate

Actively engage System 2!
● Are your ranges
○ wide? → signals your assumed distribution
○ small? → possibly you are overconfident
● Does your mental image of the distribution of data reflect the real
world?
● What is your accuracy over time?
○ are you prone to a certain bias?
○ is there an area you can learn more about to increase your accuracy?
Source:
https://hbr.org/2016/05/superforecasting-how-to-upgrade-your-companys-judgment
3. evaluate

● individuals’ forecasting ability improves already after 1 hour
● use teams boosts accuracy
● track prediction performance and provide rapid feedback
○ use past data for training
○ possibly monthly challenges
→ develop a “sanity compass” for facts
3. evaluate

4. Act
4. act

4. Action: Being data-driven
What is “being data-driven”?
4. act

4. Action: What is “being data-driven”?
4. act
scientific method
1. hypothesis
2. prediction
3. test or falsify
Kill the HiPPO
X
high data quality
● relevant
● trustworthy
access to data
● you need
● when you need it
Source:
https://www.iconspng.com/image/122092/hippo-line-art

4. act
consume
numbers
(stand up, dashboard)
make argument
with
data
explore and use
data for
decision
4. Action: What is “being data-driven”?

4. Action: Old school data-driven
4. act
Meteorologists & Intelligence Services
● they are sometimes wrong
● everybody has access to data
● people talk about data
○ standardized metrics
○ similar or common understanding of core data concepts

4. Step 1: Become data literate
4. act
data literacy
the ability to understand, use and communicate data
effectively

4. Step 1: Become data literate: vanity vs. reality
4. act
Metric Definition Relevant Context Actionable
Distance km maintenance no
Time h maintenance no
Speed km/h travel yes
A B
Source:
https://www.iconfinder.com/icons/285810/auto_automobile_car_sedan_vehicle_icon#size=128

4. Step 1: Become data literate: OMTM
4. act
Metric Definition Relevant Context Actionable
Speed km/h travel yes
A B
focus on
One Metric That Matters

4. Step 1: Become data literate: Metric vs. KPI
4. act
metric
● relevant number
● not always actionable
● meta information
KPI
● measures core task
● has a target (even if only
“line in the sand”)

4. Step 1: Become data literate: Ratios
4. act
A good metric or KPI is a ratio
● actionable
○ accelerate, brake
● inherently comparative
○ WoW, MoM, YoY: sudden spike or trend?
● allows for comparing “opposing” facts
○ speeding tickets/km
150 km/h
2018-04-13
average speed

4. Step 2: Act on data
4. act
A good metric or KPI changes
the way you work

4. Step 2: Act on data: daily or weekly checklist
4. act
1. What are my most important tasks?
2. What am I and my team working to achieve (OKRs)?
3. What is my One Metric that Matters?
4. What is my KPI and what is the target?
5. What is my hypothesis?
action
target feedbackunderstand

Recap & Wrap Up
1. Define
a. data vs. insights
2. Disassemble
a. quantitative vs. qualitative
b. known vs. unknown
c. context matters
3. Evaluate
a. What You See Is All There Is
b. have hypothesis
c. make and falsify your own prediction
4. Act
a. One Metric That Matters
b. KPI = metric + target
c. ratios are your friend

Thank you!

Questions?

Working with Data Workshop

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a Working with Data Workshop

Semelhante a Working with Data Workshop (20)

Último

Último (20)

Working with Data Workshop

Notas do Editor