The talk I gave at Uncubed NYC 2015. Goes over the different dimensions of a data science project, and shows the entire process through the example of a passion project: Movie vs Movie.
What are your top ten favorite movies of all time? This is a very difficult question. But why? I explain the challenges of measuring how much we like movies, books, songs, or products; combining insights from diverse sources like the Netflix Prize, Duncan Watts' social experiments, or the beginnings of Facebook. The better we get at measuring and ranking levels of enjoyment, the better we can customize websites, sort search results, find other people with similar tastes, and recommend products, so can we overcome these challenges? Drumroll... Yes, we can.
While Movie vs Movie answers a personal question I'm passionate about, it gives a lot of insights for the entertainment industry, and the backbone process for answering business questions is the same.
73. How did they do it?
Before:
Solid assumptions
You have a certain taste.
Your taste dictates a hidden rating for Book of Eli.
When you watch it, this rating is revealed to you.
74. How did they do it?
Before:
Solid assumptions
You have a certain taste.
Your taste dictates a hidden rating for Book of Eli.
When you watch it, this rating is revealed to you.
WRONG
75. How did they do it?
After:
Your rating changes with time.
76. How did they do it?
After:
Your rating changes with time.
It depends on...
77. How did they do it?
After:
Your rating changes with time.
It depends on...
how many you rated that day
your average rating for the day
which movies you rated on this day
shown Netflix prediction
78. Y. Koren, The BellKor Solution to the Netflix Grand Prize. 2009
Trivial: Mean score of everyone
Error: 1.0540 stars
Cinematch
Error: 0.9525 stars
79. Y. Koren, The BellKor Solution to the Netflix Grand Prize. 2009
Trivial: Mean score of everyone
Error: 1.0540 stars
Cinematch
Error: 0.9525 stars
Your time dependent rating tendencies
80. Trivial: Mean score of everyone
Error: 1.0540 stars
Cinematch
Error: 0.9525 stars
Your time dependent rating tendencies
Error: 0.9278 stars
Y. Koren, The BellKor Solution to the Netflix Grand Prize. 2009
81. Trivial: Mean score of everyone
Error: 1.0540 stars
Cinematch
Error: 0.9525 stars
Your time dependent rating tendencies
Error: 0.9278 stars
Y. Koren, The BellKor Solution to the Netflix Grand Prize. 2009
12.0%
82. Trivial: Mean score of everyone
Error: 1.0540 stars
Cinematch
Error: 0.9525 stars
Your time dependent rating tendencies
Error: 0.9278 stars
without looking at which movies you like/hate!
Y. Koren, The BellKor Solution to the Netflix Grand Prize. 2009
12.0%
84. What does this suggest?
We cannot compare a movie with all others we've seen.
85. What does this suggest?
We cannot compare a movie with all others we've seen.
We compare it to a limited set.
86. What does this suggest?
We cannot compare a movie with all others we've seen.
We compare it to a limited set.
Liking (real time & remembered) depends on time and
mood.
87. What does this suggest?
We cannot compare a movie with all others we've seen.
We compare it to a limited set.
Liking (real time & remembered) depends on time and
mood.
Other people's opinions affect our own (followers / hipsters)
88. What does this suggest?
We cannot compare Book of Eli with all movies we've seen.
We compare it to a limited set.
Liking (real time & remembered) depends on time and
mood.
Other people's opinions affect our own (followers / hipsters)
90. An experiment
Same website: Music download and rating
M.J. Salganik, P.S. Dodds, D.J. Watts. Science, 311:854-856, 2006
91. An experiment
Music Lab: A website for downloading music
Alternative A:
Other people's ratings invisible
92. An experiment
Music Lab: A website for downloading music
Alternative A:
Other people's ratings invisible
More or less equal ratings
93. An experiment
Music Lab: A website for downloading music
Alternative A:
Other people's ratings invisible
Alternative B:
All ratings visible
More or less equal ratings
94. An experiment
Music Lab: A website for downloading music
Alternative A:
Other people's ratings invisible
Alternative B:
All ratings visible
More or less equal ratings
Several songs snowball in popularity
95. An experiment
Music Lab: A website for downloading music
Alternative A:
Other people's ratings invisible
Alternative B:
All ratings visible
More or less equal ratings
Several songs snowball in popularity
It's different songs for each trial
97. Problems with rating movies
We cannot compare a movie with all others we've seen.
We compare it to a limited set.
Liking (real time & remembered) depends on time and
mood.
Other people's opinions affect our own.
98. Degree of liking is
sensitive and vague
Amazing! Total
garbage
Tuesday 3am Sunday 12pm
99. Liking (real time & remembered) depends on time and
mood.
Other people's opinions affect our own.
Degree of liking is
sensitive and vague
100. Degree of liking is
sensitive and vague
Dependent on many other
environmental factors
besides our taste
101. We cannot compare a movie with all others we've seen.
We compare it to a limited set.
Degree of liking is
sensitive and vague
102. Degree of liking is
sensitive and vague
Difficult to describe
accurately and consistently
with a number
117. Trying to rate Star Wars
Map enjoyment
to a specific scale
1
118. Trying to rate Star Wars
Map enjoyment
to a specific scale
1
119. Trying to rate Star Wars
Map enjoyment
to a specific scale
1
120. Trying to rate Star Wars
choose corresponding rating
for this degree of liking
2
121. Trying to rate Star Wars
But we cannot keep
this entire history of
enjoyment in mind
122. Trying to rate Star Wars
But we cannot keep
this entire history of
enjoyment in mind
We fuzzily remember
a small subset
123. Trying to rate Star Wars
But we cannot keep
this entire history of
enjoyment in mind
We fuzzily remember
a small subset
We map based on this subset
124. Trying to rate Star Wars
But we cannot keep
this entire history of
enjoyment in mind
We fuzzily remember
a small subset
We map based on this subset
199. How did they do it?
After:
Your rating changes with time.
A small, constant increase
in uncertainty before each
comparison
3.5 4 4.5 5
Probability
uncertainty
200. Degree of liking is
sensitive and vague
Great! We have a system!
201.
202. I don’t want to
spend too much
time on this
How many is too many?
224. Quantifying human reactions are hard
books
songs
food
politicans
products
celebrities
tv shows
importance of issues
what to spend ‘fun’ budget on
teams in different sports
225. Degree of liking is
sensitive and vague
Amazing! Total
garbage
Tuesday 3am Sunday 12pm