More Related Content Similar to Highlights on most interesting RecSys papers - Elena Smirnova, Lowik Chanussot, Amine Benhalloum - Criteo (20) Highlights on most interesting RecSys papers - Elena Smirnova, Lowik Chanussot, Amine Benhalloum - Criteo1. Copyright © 2017 Criteo
Highlights of RecSys'17
Elena Smirnova, Amine Benhalloum, Lowik Chanussot
Criteo
25/09/2017
2. Copyright © 2017 Criteo
Introduction
• RecSys'17 was held in Como, Italy on August, 27-31
• +600 participants
• Increasing number of industry sessions
• Criteo presented 2 papers on Deep learning workshop
3. Copyright © 2017 Criteo
RecSys'17: Key topics
• Session-based recommendation: Elena
• Representation learning: Amine
• Scalability: Lowik
5. Copyright © 2017 Criteo
Session-based recommendation
Classical setup: independent user-item observations
link prediction
matrix factorization
New setup: sequences of user-item interactions in time
next event prediction
Time
6. Copyright © 2017 Criteo
Recurrent Neural Networks
RNNs for session-based recommendation introduced in 2015
Learns sequence embedding (aka internal state) that represents the sequence
of user-item interactions
Performs the same computation at each time step
Hidasi et al. Session-based Recommendations with Recurrent Neural Networks.
7. Copyright © 2017 Criteo
RecSys’17: Stronger baselines
Session-based kNN
• Find k most similar past sessions
• Cosine similarity of bit vectors
• Score items by the sum of session similarities
D. Jannach and M. Ludewig. When Recurrent Neural Networks meet the Neighborhood for Session-Based
Recommendation.
8. Copyright © 2017 Criteo
RecSys’17: Hierarchical Extension
Hierarchical RNNs model long-term user behavior across sessions
2 RNNs: user and session representation
M. Quadrana et al. PersonalizingSession-basedRecommendations with Hierarchical RecurrentNeuralNetworks.
9. Copyright © 2017 Criteo
RecSys’17: Contextual Extension
Condition RNN on contextual information (event type, timestamp)
Integrate at 2 levels:
Input and Output layers
Hidden Dynamics
E. Smirnova and F. Vasile. Contextual Sequence Modeling for Recommendation with Recurrent Neural Networks.
10. Copyright © 2017 Criteo
Wrap up
• Session-based recommendation has now it own track
• Stronger baselines have been introduced
• Multiple extensions to Recurrent Neural Networks to better model user
behavior
12. Copyright © 2017 Criteo
Representation learning
Learning to represent items, users
and their relationships in an
appropriate space (as a real valued
vector)
13. Copyright © 2017 Criteo
Representation learning
Leveraging available content
(images, descriptions, reviews …)
Helping with the cold start
problem
14. Copyright © 2017 Criteo
Leveraging content: Review texts
R. Catherine et al. Transnets: Learning to transform for recommendation
• Review texts are available, how
can we use them ?
• Learn a representation of the
review and then predict the
associated rating
15. Copyright © 2017 Criteo
Leveraging content: Item features
T. Nedelec et al. : Content2Vec: Specializing jointrepresentationsof productimagesand text for the task of product
recommendation
• How can we combine
heterogeneous product
representations ?
• Specialize feature
representations (text,
image, …) for the task of
item-item similarity and
merge them
16. Copyright © 2017 Criteo
Cold start problem: Attribute to feature mapping
• Can we use item characteristics to initialize an new item's latent
representation ?
• Learning attribute (item characteristics) to feature (latent space representation)
mapping.
D. Cohen et al. : Expediting Exploration by Attribute-to-Feature Mapping for Cold-Start Recommendation
17. Copyright © 2017 Criteo
Cold start problem: Attribute to feature mapping
• We learn a linear mapping between attribute vectors {𝒂𝒊} and latent
representation {𝒗𝒊}
𝑣𝑖 ≈ 𝑊𝑎𝑖
• For a new item 𝒋 we initialize its latent representation
𝑣𝑗
0
≈ 𝑊𝑎𝑗
D.Cohen et al. : Expediting Exploration by Attribute-to-Feature Mapping for Cold-Start Recommendation
18. Copyright © 2017 Criteo
Folding
• “Folding” effect of embedding can
lead to spurious recommendations
• Model has to take into account data
Missing Not At Random, introduce
metric to measure the severity of
folding
D. Xin et al. Folding: Why Good Models Sometimes Make Spurious Recommendations.
19. Copyright © 2017 Criteo
Wrap up
• Embed all the things !
• A Deep MultimodalApproach for Cold-start Music Recommendation(Oramas et al.)
• Comparing Neural and Attractiveness-based Visual Features for Artwork Recommendation
(Dominguez et al.)
• Translation based recommendation (He et al.)
• Use content and reviews
• InterpretableConvolutional Neural Networks with Dual Local and Global Attention for
Review Rating Prediction (Seo et al.)
• Recommendation of High Quality Representative Reviews in e-commerce (Paul et al.)
• Sequential recommendations
• Sequential User-based Recurrent Neural Network Recommendations (Donkers et al.)
21. Copyright © 2017 Criteo
Large scale constraints
Many products, many users
Online latency
Training time
Offline storage
22. Copyright © 2017 Criteo
Convolution at character level for session-based reco
Id
Name
Category
Tuan et al. 3D Convolutional Networks for Session-based Recommendation with Content Features
Item #1 Item #2 Item #n
view view basket
23. Copyright © 2017 Criteo
Convolution at character level for session-based reco
Id
Name
Category
0
a b c d … 1 2 3 4 5 6e …@? !
1
1
1 Id: 263N
Tuan et al. 3D Convolutional Networks for Session-based Recommendation with Content Features
Item #1 Item #2
view view basket
Item #n
24. Copyright © 2017 Criteo
Convolution at character level for session-based reco
Id
Name
Category
0
a b c d … 1 2 3 4 5 6e …@? !
1
1
1 Id: 263
Name: “iPhone”
Category:
“Phones/Apple”
V
N
N x 3V
Item #1
Tuan et al. 3D Convolutional Networks for Session-based Recommendation with Content Features
Item #1 Item #2
view view basket
Item #n
25. Copyright © 2017 Criteo
Convolution at character level for session-based reco
Item #1 Item #2
Id
Name
Category
0
a b c d … 1 2 3 4 5 6e …@? !
1
1
1
3*V = 56
N=150
Compact Input 3V x N x D
Id: 263
Name: “iPhone”
Category:
“Phones/Apple”
V
N
N x 3V
Item #1
Tuan et al. 3D Convolutional Networks for Session-based Recommendation with Content Features
view view basket
Item #n
26. Copyright © 2017 Criteo
Convolution at character level for session-based reco
Tuan et al. 3D Convolutional Networks for Session-based Recommendation with Content Features
27. Copyright © 2017 Criteo
Scaling deep nets
Your favorite
deep net
D x L1 D x Ln
- Compact
- Fast to compute
- For input and output
- Reversible output
- No change in the deep-net
- Appropriate loss
- Same accuracy
Serrà et al. Getting Deep Recommenders Fit : Bloom Embeddings for Sparse Binary Input / Output Networks
28. Copyright © 2017 Criteo
Bloom filters embeddings
1
1
1
H1
H2
Hk
1
D
m < D
p
Serrà et al. Getting Deep Recommenders Fit : Bloom Embeddings for Sparse Binary Input / Output Networks
29. Copyright © 2017 Criteo
Bloom filters embeddings
1
1
1
H1
H2
Hk
1
Your favorite
deep net
0.1
0.3
0.2
D
m < D
m x L1 m x Ln
p
v
Serrà et al. Getting Deep Recommenders Fit : Bloom Embeddings for Sparse Binary Input / Output Networks
30. Copyright © 2017 Criteo
Bloom filters embeddings
1
1
1
H1
H2
Hk
1
Your favorite
deep net
0.1
0.3
0.2
D
m < D
m x L1 m x Ln
1 q
p
H1
Hk
y
v
H2
Serrà et al. Getting Deep Recommenders Fit : Bloom Embeddings for Sparse Binary Input / Output Networks
31. Copyright © 2017 Criteo
Wrap up
Dedicated workshop
Preselection of products
Scaling deep nets
Online
Ranking
Candidates
Items
Offline
Selection
N products
K products (K << N)
32. Copyright © 2017 Criteo
Conclusion
RecSys’18 - Vancouver, Canada, 2nd-7th October 2018