Quantitative measurement is the key to scaling businesses, processes, and products and making them better. It sounds easy: just pick a number, and improve it. It's not.
Choosing a metric is an exploration of a many-dimensional space with no map and no guide. Until now. This talk will teach you the science of choosing metrics that will guide you to building a better product. When you attend this webinar, you will:
Learn how applying data is as much about questions as it is about answers
Discover the trade-offs between different metrics
Understand systematic ways to measure your product funnel
Learn how to isolate the most important metrics to maximize your product impact
40. OR
More Jobs
Data Scientist
Microsoft
4,230 reviews - London
The role
We are looking to growing our
data team and if you love
understanding the customers
through the data lens this
position is for you. We are
poised with finding new ways for
uncovering data insights to
increase our ability to build an
More Resumes
President
president@indeed.com
Work Experience
President
Indeed - Austin, TX
Oct. 2015 -Present
Indeed is the #1 job site
worldwide with over 180
million unique visitors and 2
billion searches per month
in more than 60 countries.
SVP, Product & Engineering
Indeed - Austin, TX
October 2010 - October 2015
Indeed is the #1 job site
worldwide with over 180
million unique visitors and 2
billion searches per month
in more than 60 countries.
41. More Jobs More Resumes
President
president@indeed.com
Work Experience
President
Indeed - Austin, TX
Oct. 2015 -Present
Indeed is the #1 job site
worldwide with over 180
million unique visitors and 2
billion searches per month
in more than 60 countries.
SVP, Product & Engineering
Indeed - Austin, TX
October 2010 - October 2015
Indeed is the #1 job site
worldwide with over 180
million unique visitors and 2
billion searches per month
in more than 60 countries.
Lifetime Value
Data Scientist
Microsoft
4,230 reviews - London
The role
We are looking to growing our
data team and if you love
understanding the customers
through the data lens this
position is for you. We are
poised with finding new ways for
uncovering data insights to
increase our ability to build an
82. + Exploratory product discovering possible outcomes
+ Opportunities for big improvements
+ Only simple analysis
+ Less influence over outcome
Favor outputs
89. I very frequently get the question: "What’s
going to change in the next 10 years?"
I almost never get the question: "What’s not
going to change in the next 10 years?"
That second question is actually the more
important of the two — because you can
build a business strategy around the things
that are stable in time…
We know that customers want low prices, and
I know that’s going to be true 10 years from
now. They want fast delivery; they want vast
selection.
It’s impossible to imagine a future 10 years
from now where a customer comes up and
says, "Jeff I love Amazon; I just wish the
prices were a little higher," [or] "I love
Amazon; I just wish you’d deliver a little more
slowly."
Jeff Bezos
Chairman, CEO, President, and Ruthless Cyborg Killer
Amazon, Inc.
115. Measuring a funnel
1 Quality of input
2 Quantity of input
3 Time to complete
4 Effort to complete
5 Cost
6 Happiness
7 Number of stages
8 Quality of outcome
9 Quantity of outcome
116. So many choices…
1 Quality of input
2 Quantity of input
3 Quality of output
4 Quantity of output
5 Time
6 Effort
7 Cost
8 Happiness
1 Quality of input
2 Quantity of input
3 Time to complete
4 Effort to complete
5 Cost
6 Happiness
7 Number of stages
8 Quality of outcome
9 Quantity of outcome
149. + Insincere activity
+ Indiscriminate targeting of jobs
Optimizing for phone screens
Encourages
+ Deeper engagement
Discourages
+ Broad attention across many jobs
+ Selective targeting of jobs
+ Many applications to fewer jobs
Is this what we want?
163. 1) Estimate sensitivity and correlation
2) Find your Pareto frontier
3) Map your funnel and apply standard metrics
4) Trade off outputs and outcomes
5) Target 3-4 independent metrics
6) Run thought experiments
165. More information
Contact ketan@scienceofmetrics.com
Video of v1 of this talk https://go.indeed.com/alchemy-metrics
Key points summary https://go.indeed.com/MetricsThatWork
Indeed Engineering Blog http://engineering.indeedblog.com/blog/
166. Next Session
How to Find Your Product's North Star Metric
Presented by Alicia Shiu, Dropbox