Trailers Help Reduce Popularity Bias in Choice Recommenders

Can Trailers Help to
Alleviate Popularity Bias in
Choice-Based Preference
Elicitation?
Mark P. Graus
Martijn C. Willemsen
Human-Technology
Interaction Group
Eindhoven University
of Technology

Summary
• We wanted to see if we could make people chose
less popular items in a choice-based preference
elicitation recommender system by showing them
trailers.
• We tested this in a between subjects user study.
• We found that after watching trailers people chose
less popular items, while user experience was not
negatively affected.
9/16/2016
Graus, Willemsen: Can Trailers Help to Alleviate ... IntRS @
RecSys '16
2

Motivation
9/16/2016
RecSys '16
3

Latent Feature Diversification
• Can we reduce choice overload
through diversification based on
the latent features of a matrix
factorization model?
Willemsen, M. C., Graus, M. P., & Knijnenburg, B.
P. (2016). Understanding the role of latent feature
diversification on choice difficulty and satisfaction.
User Modeling and User-Adapted Interaction, 1–
43. http://doi.org/10.1007/s11257-016-9178-6
Latent Feature 1
LatentFeature2
9/16/2016
RecSys '16
4

Latent Feature Diversification Findings
9/16/2016
RecSys '16
5
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
low mid high
standardizedscore
diversification
Perceived diversity
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
low mid high
standardizedscore
diversification
Expected choice difficulty

LatentFeature2
Latent Feature 1
4) Iteration 2
Choice-Based Preference Elicitation
• Can we improve the user experience
during cold start by having people
choose between items instead of
rating items?
Graus, M. P., & Willemsen, M. C. (2015). Improving the
User Experience during Cold Start through Choice-
Based Preference Elicitation. In Proceedings of the 9th
ACM Conference on Recommender Systems - RecSys
’15 (pp. 273–276). New York, New York, USA: ACM
Press. http://doi.org/10.1145/2792838.2799681
9/16/2016
RecSys '16
6

How does this work? Step 1
Latent Feature 1
LatentFeature2
Iteration 1a: Diversified choice set is
calculated from a matrix factorization
model (red items)
Iteration 1b: User vector (blue arrow) is moved
towards chosen item (green item), items with
lowest predicted rating are discarded (greyed
out items)
9/16/2016
RecSys '16
7

How does this work? Step 2
Iteration 2: New diversified choice set
(blue items)
End of Iteration 2: with updated vector and
more items discarded based on second choice
(green item)
9/16/2016
RecSys '16
8

Choice-Based Preference Elicitation
Findings
• People are more satisfied with choice-based than
rating-based interfaces
• This comes mainly because of increased popularity
(items with many ratings)
But we do not want to
recommend popular
items!
9/16/2016
RecSys '16
9
Satisfaction
with Chosen
Item
Popularity
Difficulty
Intra List
Similarity
-2.407(.381)
p<.001
-.240 (.145)
p<.1
-.479 (.111)
p<.001
-.257 (.045)
p<.001
14.00 (4.51)
p<.01
Choice-
Based List
+
+
- -
+

Why do people end up with popular
items?
• Our hypothesis
• Users don’t know all movies, hard to judge based on
metadata alone
• People choose movies they know
• People know movies that are popular
• Choosing popular movies results in popular recommendations
• Our Solution
• Provide trailers as additional information for making choices
9/16/2016
RecSys '16
10

Rationale
• In the music domain
• Implicit Feedback
• Movie domain
• Implicit feedback is sparse
• I (can) listen to 100s of tracks in a week, but I can’t
watch 100s of movies a week (and sustain my job).
• We can approximate experiencing movies by
providing trailers
9/16/2016
RecSys '16
11

Study
9/16/2016
RecSys '16
12

Choice-Based Interface with or without
Trailers
• N = 71
9/16/2016
RecSys '16
13

Expected Effects
Trailers
Perceived
Diversity
Informativeness
Perceived
Novelty
Choice
Satisfaction
System
Satisfaction
Popularity of
Chosen Items
- +
+
+?
-?-
-
+
9/16/2016
RecSys '16
14

Set Up
• Random assignment
• Choice Based Preference Elicitation – 9 choices of 10 items
[with/without trailers]
• Recommendation List – Top-10 Items [with/without trailers]
• Survey to measure User Experience
• Informativeness
• Perceived Diversity
• Perceived Novelty
• System Satisfaction
• Choice Satisfaction
9/16/2016
RecSys '16
15

Results
9/16/2016
RecSys '16
16

Do trailers affect the popularity of chosen
items?
• Checked through repeated measures (10 choices)
• Popularity is expressed as the rank ordering by
number of ratings in MovieLens dataset
• Trailers do not decrease popularity of choices
• The popularity rank of the item chosen in each choice
set
• The average popularity rank of all items in each choice
set
9/16/2016
RecSys '16
17

However: Relative Popularity of Choice
• average popularity rank of choice set – popularity
rank of chosen item
9/16/2016
RecSys '16
18

If we look at people that actually watched
trailers
• People that watch trailers are
more likely to pick less popular
movies from the lists
9/16/2016
RecSys '16
19

User Experience
Choice
Satisfaction
Perceived
Diversity
System
Satisfaction
Informativeness
.570 (.295)
p < 0.1
-.604 (.091)
p < 0.01
.244 (.162)
n.s.
.785 (.115)
p < 0.01
-.266 (.122)
p < 0.05
Trailers
.611 (.256)
p < 0.05
-.570 (.259)
p < 0.05
-
+
-
-
+
9/16/2016
+
RecSys '16
20

Conclusions
9/16/2016
RecSys '16
21

What we found
• Providing people with trailers does make them
choose less popular items.
• No indication that the overall satisfaction is affected
negatively or positively
• As opposed to initial study where popularity resulted in
increased satisfaction
9/16/2016
RecSys '16
22

Limitations
• When were trailers watched?
• In the preference elicitation task?
• In the decision task?
Future Work
• How do trailers affect a more standard rating
interface?
9/16/2016
RecSys '16
23

Thank You
• Questions/remarks?
Mark Graus – PhD Student
Human-Technology Interaction Group
Eindhoven University of Technology
m.p.graus@tue.nl
https://twitter.com/newmarrk
https://linkedin.com/in/markgraus
http://www.marrk.nl
9/16/2016
RecSys '16
24

Trailers Help Reduce Popularity Bias in Choice Recommenders

Recomendados

Recomendados

Mais conteúdo relacionado

Destaque

Destaque (12)

Último

Último (20)

Trailers Help Reduce Popularity Bias in Choice Recommenders