SlideShare a Scribd company logo
1 of 34
Download to read offline
Copyright © 2017 Criteo
Highlights of RecSys'17
Elena Smirnova, Amine Benhalloum, Lowik Chanussot
Criteo
25/09/2017
Copyright © 2017 Criteo
Introduction
• RecSys'17 was held in Como, Italy on August, 27-31
• +600 participants
• Increasing number of industry sessions
• Criteo presented 2 papers on Deep learning workshop
Copyright © 2017 Criteo
RecSys'17: Key topics
• Session-based recommendation: Elena
• Representation learning: Amine
• Scalability: Lowik
Copyright © 2017 Criteo
Session-based
recommendation
Copyright © 2017 Criteo
Session-based recommendation
Classical setup: independent user-item observations
link prediction
matrix factorization
New setup: sequences of user-item interactions in time
next event prediction
Time
Copyright © 2017 Criteo
Recurrent Neural Networks
RNNs for session-based recommendation introduced in 2015
Learns sequence embedding (aka internal state) that represents the sequence
of user-item interactions
Performs the same computation at each time step
Hidasi et al. Session-based Recommendations with Recurrent Neural Networks.
Copyright © 2017 Criteo
RecSys’17: Stronger baselines
Session-based kNN
• Find k most similar past sessions
• Cosine similarity of bit vectors
• Score items by the sum of session similarities
D. Jannach and M. Ludewig. When Recurrent Neural Networks meet the Neighborhood for Session-Based
Recommendation.
Copyright © 2017 Criteo
RecSys’17: Hierarchical Extension
Hierarchical RNNs model long-term user behavior across sessions
2 RNNs: user and session representation
M. Quadrana et al. PersonalizingSession-basedRecommendations with Hierarchical RecurrentNeuralNetworks.
Copyright © 2017 Criteo
RecSys’17: Contextual Extension
Condition RNN on contextual information (event type, timestamp)
Integrate at 2 levels:
Input and Output layers
Hidden Dynamics
E. Smirnova and F. Vasile. Contextual Sequence Modeling for Recommendation with Recurrent Neural Networks.
Copyright © 2017 Criteo
Wrap up
• Session-based recommendation has now it own track
• Stronger baselines have been introduced
• Multiple extensions to Recurrent Neural Networks to better model user
behavior
Copyright © 2017 Criteo
Representation learning
Copyright © 2017 Criteo
Representation learning
Learning to represent items, users
and their relationships in an
appropriate space (as a real valued
vector)
Copyright © 2017 Criteo
Representation learning
Leveraging available content
(images, descriptions, reviews …)
Helping with the cold start
problem
Copyright © 2017 Criteo
Leveraging content: Review texts
R. Catherine et al. Transnets: Learning to transform for recommendation
• Review texts are available, how
can we use them ?
• Learn a representation of the
review and then predict the
associated rating
Copyright © 2017 Criteo
Leveraging content: Item features
T. Nedelec et al. : Content2Vec: Specializing jointrepresentationsof productimagesand text for the task of product
recommendation
• How can we combine
heterogeneous product
representations ?
• Specialize feature
representations (text,
image, …) for the task of
item-item similarity and
merge them
Copyright © 2017 Criteo
Cold start problem: Attribute to feature mapping
• Can we use item characteristics to initialize an new item's latent
representation ?
• Learning attribute (item characteristics) to feature (latent space representation)
mapping.
D. Cohen et al. : Expediting Exploration by Attribute-to-Feature Mapping for Cold-Start Recommendation
Copyright © 2017 Criteo
Cold start problem: Attribute to feature mapping
• We learn a linear mapping between attribute vectors {𝒂𝒊} and latent
representation {𝒗𝒊}
𝑣𝑖 ≈ 𝑊𝑎𝑖
• For a new item 𝒋 we initialize its latent representation
𝑣𝑗
0
≈ 𝑊𝑎𝑗
D.Cohen et al. : Expediting Exploration by Attribute-to-Feature Mapping for Cold-Start Recommendation
Copyright © 2017 Criteo
Folding
• “Folding” effect of embedding can
lead to spurious recommendations
• Model has to take into account data
Missing Not At Random, introduce
metric to measure the severity of
folding
D. Xin et al. Folding: Why Good Models Sometimes Make Spurious Recommendations.
Copyright © 2017 Criteo
Wrap up
• Embed all the things !
• A Deep MultimodalApproach for Cold-start Music Recommendation(Oramas et al.)
• Comparing Neural and Attractiveness-based Visual Features for Artwork Recommendation
(Dominguez et al.)
• Translation based recommendation (He et al.)
• Use content and reviews
• InterpretableConvolutional Neural Networks with Dual Local and Global Attention for
Review Rating Prediction (Seo et al.)
• Recommendation of High Quality Representative Reviews in e-commerce (Paul et al.)
• Sequential recommendations
• Sequential User-based Recurrent Neural Network Recommendations (Donkers et al.)
Copyright © 2017 Criteo
Scalability
Copyright © 2017 Criteo
Large scale constraints
Many products, many users
Online latency
Training time
Offline storage
Copyright © 2017 Criteo
Convolution at character level for session-based reco
Id
Name
Category
Tuan et al. 3D Convolutional Networks for Session-based Recommendation with Content Features
Item #1 Item #2 Item #n
view view basket
Copyright © 2017 Criteo
Convolution at character level for session-based reco
Id
Name
Category
0
a b c d … 1 2 3 4 5 6e …@? !
1
1
1 Id: 263N
Tuan et al. 3D Convolutional Networks for Session-based Recommendation with Content Features
Item #1 Item #2
view view basket
Item #n
Copyright © 2017 Criteo
Convolution at character level for session-based reco
Id
Name
Category
0
a b c d … 1 2 3 4 5 6e …@? !
1
1
1 Id: 263
Name: “iPhone”
Category:
“Phones/Apple”
V
N
N x 3V
Item #1
Tuan et al. 3D Convolutional Networks for Session-based Recommendation with Content Features
Item #1 Item #2
view view basket
Item #n
Copyright © 2017 Criteo
Convolution at character level for session-based reco
Item #1 Item #2
Id
Name
Category
0
a b c d … 1 2 3 4 5 6e …@? !
1
1
1
3*V = 56
N=150
Compact Input 3V x N x D
Id: 263
Name: “iPhone”
Category:
“Phones/Apple”
V
N
N x 3V
Item #1
Tuan et al. 3D Convolutional Networks for Session-based Recommendation with Content Features
view view basket
Item #n
Copyright © 2017 Criteo
Convolution at character level for session-based reco
Tuan et al. 3D Convolutional Networks for Session-based Recommendation with Content Features
Copyright © 2017 Criteo
Scaling deep nets
Your favorite
deep net
D x L1 D x Ln
- Compact
- Fast to compute
- For input and output
- Reversible output
- No change in the deep-net
- Appropriate loss
- Same accuracy
Serrà et al. Getting Deep Recommenders Fit : Bloom Embeddings for Sparse Binary Input / Output Networks
Copyright © 2017 Criteo
Bloom filters embeddings
1
1
1
H1
H2
Hk
1
D
m < D
p
Serrà et al. Getting Deep Recommenders Fit : Bloom Embeddings for Sparse Binary Input / Output Networks
Copyright © 2017 Criteo
Bloom filters embeddings
1
1
1
H1
H2
Hk
1
Your favorite
deep net
0.1
0.3
0.2
D
m < D
m x L1 m x Ln
p
v
Serrà et al. Getting Deep Recommenders Fit : Bloom Embeddings for Sparse Binary Input / Output Networks
Copyright © 2017 Criteo
Bloom filters embeddings
1
1
1
H1
H2
Hk
1
Your favorite
deep net
0.1
0.3
0.2
D
m < D
m x L1 m x Ln
1 q
p
H1
Hk
y
v
H2
Serrà et al. Getting Deep Recommenders Fit : Bloom Embeddings for Sparse Binary Input / Output Networks
Copyright © 2017 Criteo
Wrap up
Dedicated workshop
Preselection of products
Scaling deep nets
Online
Ranking
Candidates
Items
Offline
Selection
N products
K products (K << N)
Copyright © 2017 Criteo
Conclusion
RecSys’18 - Vancouver, Canada, 2nd-7th October 2018
Copyright © 2017 Criteo
Thank you!
Copyright © 2017 Criteo
Join Criteo!

More Related Content

Similar to Highlights on most interesting RecSys papers - Elena Smirnova, Lowik Chanussot, Amine Benhalloum - Criteo

Efficient Data Formats for Analytics with Parquet and Arrow
Efficient Data Formats for Analytics with Parquet and ArrowEfficient Data Formats for Analytics with Parquet and Arrow
Efficient Data Formats for Analytics with Parquet and Arrow
DataWorks Summit/Hadoop Summit
 

Similar to Highlights on most interesting RecSys papers - Elena Smirnova, Lowik Chanussot, Amine Benhalloum - Criteo (20)

2017 09-20-criteo-recsys-london
2017 09-20-criteo-recsys-london2017 09-20-criteo-recsys-london
2017 09-20-criteo-recsys-london
 
Building Large Sustainable Apps
Building Large Sustainable AppsBuilding Large Sustainable Apps
Building Large Sustainable Apps
 
Efficient Data Formats for Analytics with Parquet and Arrow
Efficient Data Formats for Analytics with Parquet and ArrowEfficient Data Formats for Analytics with Parquet and Arrow
Efficient Data Formats for Analytics with Parquet and Arrow
 
Steeltoe and the Open Source .NET Renaissance
Steeltoe and the Open Source .NET RenaissanceSteeltoe and the Open Source .NET Renaissance
Steeltoe and the Open Source .NET Renaissance
 
TCUK 2012, Nolwenn Kerzreho, Metadata: Why Should Technical Communicators Care?
TCUK 2012, Nolwenn Kerzreho, Metadata: Why Should Technical Communicators Care?TCUK 2012, Nolwenn Kerzreho, Metadata: Why Should Technical Communicators Care?
TCUK 2012, Nolwenn Kerzreho, Metadata: Why Should Technical Communicators Care?
 
GluonCV
GluonCVGluonCV
GluonCV
 
Modular Documentation Joe Gelb Techshoret 2009
Modular Documentation Joe Gelb Techshoret 2009Modular Documentation Joe Gelb Techshoret 2009
Modular Documentation Joe Gelb Techshoret 2009
 
Elastic @ Adobe: Making Search Smarter with Machine Learning at Scale
Elastic @ Adobe: Making Search Smarter with Machine Learning at ScaleElastic @ Adobe: Making Search Smarter with Machine Learning at Scale
Elastic @ Adobe: Making Search Smarter with Machine Learning at Scale
 
Tensors for topic modeling and deep learning on AWS Sagemaker
Tensors for topic modeling and deep learning on AWS SagemakerTensors for topic modeling and deep learning on AWS Sagemaker
Tensors for topic modeling and deep learning on AWS Sagemaker
 
Charting libraries vs BI & Analytics Platforms
Charting libraries vs BI & Analytics Platforms Charting libraries vs BI & Analytics Platforms
Charting libraries vs BI & Analytics Platforms
 
Kentico Cloud and .NET Core - A Winning Combination
Kentico Cloud and .NET Core - A Winning CombinationKentico Cloud and .NET Core - A Winning Combination
Kentico Cloud and .NET Core - A Winning Combination
 
Introduction to Domain-Driven Design
Introduction to Domain-Driven DesignIntroduction to Domain-Driven Design
Introduction to Domain-Driven Design
 
The art of decomposing monoliths - Kfir Bloch - Codemotion Amsterdam 2016
The art of decomposing monoliths - Kfir Bloch - Codemotion Amsterdam 2016The art of decomposing monoliths - Kfir Bloch - Codemotion Amsterdam 2016
The art of decomposing monoliths - Kfir Bloch - Codemotion Amsterdam 2016
 
Repurpose, Reuse and Refresh Content
Repurpose, Reuse and Refresh ContentRepurpose, Reuse and Refresh Content
Repurpose, Reuse and Refresh Content
 
Domain Driven Design - Building Blocks
Domain Driven Design - Building BlocksDomain Driven Design - Building Blocks
Domain Driven Design - Building Blocks
 
Understanding and Addressing Architectural Challenges of Cloud- Based Systems
Understanding and Addressing Architectural Challenges of Cloud- Based SystemsUnderstanding and Addressing Architectural Challenges of Cloud- Based Systems
Understanding and Addressing Architectural Challenges of Cloud- Based Systems
 
Sentiment analysis in Twitter on Big Data
Sentiment analysis in Twitter on Big DataSentiment analysis in Twitter on Big Data
Sentiment analysis in Twitter on Big Data
 
Apache Arrow: In Theory, In Practice
Apache Arrow: In Theory, In PracticeApache Arrow: In Theory, In Practice
Apache Arrow: In Theory, In Practice
 
How Twilio Scaled Its Data Driven Culture - ABD309 - re:Invent 2017
How Twilio Scaled Its Data Driven Culture - ABD309 - re:Invent 2017How Twilio Scaled Its Data Driven Culture - ABD309 - re:Invent 2017
How Twilio Scaled Its Data Driven Culture - ABD309 - re:Invent 2017
 
Big Data LDN 2017: How Big Data Insights Become Easily Accessible With Workfl...
Big Data LDN 2017: How Big Data Insights Become Easily Accessible With Workfl...Big Data LDN 2017: How Big Data Insights Become Easily Accessible With Workfl...
Big Data LDN 2017: How Big Data Insights Become Easily Accessible With Workfl...
 

More from recsysfr

More from recsysfr (17)

Multi Task DPP for Basket Completion by Romain WARLOP, Fifty Five
Multi Task DPP for Basket Completion by Romain WARLOP, Fifty FiveMulti Task DPP for Basket Completion by Romain WARLOP, Fifty Five
Multi Task DPP for Basket Completion by Romain WARLOP, Fifty Five
 
Building a recommender system with Annoy and Word2Vec by Cristian PEREZ, Kern...
Building a recommender system with Annoy and Word2Vec by Cristian PEREZ, Kern...Building a recommender system with Annoy and Word2Vec by Cristian PEREZ, Kern...
Building a recommender system with Annoy and Word2Vec by Cristian PEREZ, Kern...
 
An Homophily-based Approach for Fast Post Recommendation in Microblogging Sys...
An Homophily-based Approach for Fast Post Recommendation in Microblogging Sys...An Homophily-based Approach for Fast Post Recommendation in Microblogging Sys...
An Homophily-based Approach for Fast Post Recommendation in Microblogging Sys...
 
What can bring library metadata to the web? Trust, links and love
What can bring library metadata to the web? Trust, links and loveWhat can bring library metadata to the web? Trust, links and love
What can bring library metadata to the web? Trust, links and love
 
Dictionary Learning for Massive Matrix Factorization
Dictionary Learning for Massive Matrix FactorizationDictionary Learning for Massive Matrix Factorization
Dictionary Learning for Massive Matrix Factorization
 
Recommendations @ Rakuten Group
Recommendations @ Rakuten GroupRecommendations @ Rakuten Group
Recommendations @ Rakuten Group
 
Data-Driven Recommender Systems
Data-Driven Recommender SystemsData-Driven Recommender Systems
Data-Driven Recommender Systems
 
Recommender systems
Recommender systemsRecommender systems
Recommender systems
 
Recommendation @Deezer
Recommendation @DeezerRecommendation @Deezer
Recommendation @Deezer
 
Flexible recommender systems based on graphs
Flexible recommender systems based on graphsFlexible recommender systems based on graphs
Flexible recommender systems based on graphs
 
Using Neural Networks to predict user ratings
Using Neural Networks to predict user ratingsUsing Neural Networks to predict user ratings
Using Neural Networks to predict user ratings
 
Preference Elicitation in Mangaki: Is Your Taste Kinda Weird?
Preference Elicitation in Mangaki: Is Your Taste Kinda Weird?Preference Elicitation in Mangaki: Is Your Taste Kinda Weird?
Preference Elicitation in Mangaki: Is Your Taste Kinda Weird?
 
Recommendation @ PriceMinister-Rakuten - Road to personalization
Recommendation @ PriceMinister-Rakuten - Road to personalizationRecommendation @ PriceMinister-Rakuten - Road to personalization
Recommendation @ PriceMinister-Rakuten - Road to personalization
 
Rakuten Institute of Technology Paris
Rakuten Institute of Technology ParisRakuten Institute of Technology Paris
Rakuten Institute of Technology Paris
 
Tailor-made personalization and recommendation - Sailendra
Tailor-made personalization and recommendation - SailendraTailor-made personalization and recommendation - Sailendra
Tailor-made personalization and recommendation - Sailendra
 
New tools from the bandit literature to improve A/B Testing
New tools from the bandit literature to improve A/B TestingNew tools from the bandit literature to improve A/B Testing
New tools from the bandit literature to improve A/B Testing
 
Story of the algorithms behind Deezer Flow
Story of the algorithms behind Deezer FlowStory of the algorithms behind Deezer Flow
Story of the algorithms behind Deezer Flow
 

Recently uploaded

Conjugation, transduction and transformation
Conjugation, transduction and transformationConjugation, transduction and transformation
Conjugation, transduction and transformation
Areesha Ahmad
 
Bacterial Identification and Classifications
Bacterial Identification and ClassificationsBacterial Identification and Classifications
Bacterial Identification and Classifications
Areesha Ahmad
 
Digital Dentistry.Digital Dentistryvv.pptx
Digital Dentistry.Digital Dentistryvv.pptxDigital Dentistry.Digital Dentistryvv.pptx
Digital Dentistry.Digital Dentistryvv.pptx
MohamedFarag457087
 
Module for Grade 9 for Asynchronous/Distance learning
Module for Grade 9 for Asynchronous/Distance learningModule for Grade 9 for Asynchronous/Distance learning
Module for Grade 9 for Asynchronous/Distance learning
levieagacer
 
The Mariana Trench remarkable geological features on Earth.pptx
The Mariana Trench remarkable geological features on Earth.pptxThe Mariana Trench remarkable geological features on Earth.pptx
The Mariana Trench remarkable geological features on Earth.pptx
seri bangash
 

Recently uploaded (20)

9999266834 Call Girls In Noida Sector 22 (Delhi) Call Girl Service
9999266834 Call Girls In Noida Sector 22 (Delhi) Call Girl Service9999266834 Call Girls In Noida Sector 22 (Delhi) Call Girl Service
9999266834 Call Girls In Noida Sector 22 (Delhi) Call Girl Service
 
Clean In Place(CIP).pptx .
Clean In Place(CIP).pptx                 .Clean In Place(CIP).pptx                 .
Clean In Place(CIP).pptx .
 
High Class Escorts in Hyderabad ₹7.5k Pick Up & Drop With Cash Payment 969456...
High Class Escorts in Hyderabad ₹7.5k Pick Up & Drop With Cash Payment 969456...High Class Escorts in Hyderabad ₹7.5k Pick Up & Drop With Cash Payment 969456...
High Class Escorts in Hyderabad ₹7.5k Pick Up & Drop With Cash Payment 969456...
 
Connaught Place, Delhi Call girls :8448380779 Model Escorts | 100% verified
Connaught Place, Delhi Call girls :8448380779 Model Escorts | 100% verifiedConnaught Place, Delhi Call girls :8448380779 Model Escorts | 100% verified
Connaught Place, Delhi Call girls :8448380779 Model Escorts | 100% verified
 
Conjugation, transduction and transformation
Conjugation, transduction and transformationConjugation, transduction and transformation
Conjugation, transduction and transformation
 
PSYCHOSOCIAL NEEDS. in nursing II sem pptx
PSYCHOSOCIAL NEEDS. in nursing II sem pptxPSYCHOSOCIAL NEEDS. in nursing II sem pptx
PSYCHOSOCIAL NEEDS. in nursing II sem pptx
 
Vip profile Call Girls In Lonavala 9748763073 For Genuine Sex Service At Just...
Vip profile Call Girls In Lonavala 9748763073 For Genuine Sex Service At Just...Vip profile Call Girls In Lonavala 9748763073 For Genuine Sex Service At Just...
Vip profile Call Girls In Lonavala 9748763073 For Genuine Sex Service At Just...
 
Site Acceptance Test .
Site Acceptance Test                    .Site Acceptance Test                    .
Site Acceptance Test .
 
Bacterial Identification and Classifications
Bacterial Identification and ClassificationsBacterial Identification and Classifications
Bacterial Identification and Classifications
 
FAIRSpectra - Enabling the FAIRification of Analytical Science
FAIRSpectra - Enabling the FAIRification of Analytical ScienceFAIRSpectra - Enabling the FAIRification of Analytical Science
FAIRSpectra - Enabling the FAIRification of Analytical Science
 
Digital Dentistry.Digital Dentistryvv.pptx
Digital Dentistry.Digital Dentistryvv.pptxDigital Dentistry.Digital Dentistryvv.pptx
Digital Dentistry.Digital Dentistryvv.pptx
 
COST ESTIMATION FOR A RESEARCH PROJECT.pptx
COST ESTIMATION FOR A RESEARCH PROJECT.pptxCOST ESTIMATION FOR A RESEARCH PROJECT.pptx
COST ESTIMATION FOR A RESEARCH PROJECT.pptx
 
Proteomics: types, protein profiling steps etc.
Proteomics: types, protein profiling steps etc.Proteomics: types, protein profiling steps etc.
Proteomics: types, protein profiling steps etc.
 
Module for Grade 9 for Asynchronous/Distance learning
Module for Grade 9 for Asynchronous/Distance learningModule for Grade 9 for Asynchronous/Distance learning
Module for Grade 9 for Asynchronous/Distance learning
 
Grade 7 - Lesson 1 - Microscope and Its Functions
Grade 7 - Lesson 1 - Microscope and Its FunctionsGrade 7 - Lesson 1 - Microscope and Its Functions
Grade 7 - Lesson 1 - Microscope and Its Functions
 
Forensic Biology & Its biological significance.pdf
Forensic Biology & Its biological significance.pdfForensic Biology & Its biological significance.pdf
Forensic Biology & Its biological significance.pdf
 
Justdial Call Girls In Indirapuram, Ghaziabad, 8800357707 Escorts Service
Justdial Call Girls In Indirapuram, Ghaziabad, 8800357707 Escorts ServiceJustdial Call Girls In Indirapuram, Ghaziabad, 8800357707 Escorts Service
Justdial Call Girls In Indirapuram, Ghaziabad, 8800357707 Escorts Service
 
High Profile 🔝 8250077686 📞 Call Girls Service in GTB Nagar🍑
High Profile 🔝 8250077686 📞 Call Girls Service in GTB Nagar🍑High Profile 🔝 8250077686 📞 Call Girls Service in GTB Nagar🍑
High Profile 🔝 8250077686 📞 Call Girls Service in GTB Nagar🍑
 
Pulmonary drug delivery system M.pharm -2nd sem P'ceutics
Pulmonary drug delivery system M.pharm -2nd sem P'ceuticsPulmonary drug delivery system M.pharm -2nd sem P'ceutics
Pulmonary drug delivery system M.pharm -2nd sem P'ceutics
 
The Mariana Trench remarkable geological features on Earth.pptx
The Mariana Trench remarkable geological features on Earth.pptxThe Mariana Trench remarkable geological features on Earth.pptx
The Mariana Trench remarkable geological features on Earth.pptx
 

Highlights on most interesting RecSys papers - Elena Smirnova, Lowik Chanussot, Amine Benhalloum - Criteo

  • 1. Copyright © 2017 Criteo Highlights of RecSys'17 Elena Smirnova, Amine Benhalloum, Lowik Chanussot Criteo 25/09/2017
  • 2. Copyright © 2017 Criteo Introduction • RecSys'17 was held in Como, Italy on August, 27-31 • +600 participants • Increasing number of industry sessions • Criteo presented 2 papers on Deep learning workshop
  • 3. Copyright © 2017 Criteo RecSys'17: Key topics • Session-based recommendation: Elena • Representation learning: Amine • Scalability: Lowik
  • 4. Copyright © 2017 Criteo Session-based recommendation
  • 5. Copyright © 2017 Criteo Session-based recommendation Classical setup: independent user-item observations link prediction matrix factorization New setup: sequences of user-item interactions in time next event prediction Time
  • 6. Copyright © 2017 Criteo Recurrent Neural Networks RNNs for session-based recommendation introduced in 2015 Learns sequence embedding (aka internal state) that represents the sequence of user-item interactions Performs the same computation at each time step Hidasi et al. Session-based Recommendations with Recurrent Neural Networks.
  • 7. Copyright © 2017 Criteo RecSys’17: Stronger baselines Session-based kNN • Find k most similar past sessions • Cosine similarity of bit vectors • Score items by the sum of session similarities D. Jannach and M. Ludewig. When Recurrent Neural Networks meet the Neighborhood for Session-Based Recommendation.
  • 8. Copyright © 2017 Criteo RecSys’17: Hierarchical Extension Hierarchical RNNs model long-term user behavior across sessions 2 RNNs: user and session representation M. Quadrana et al. PersonalizingSession-basedRecommendations with Hierarchical RecurrentNeuralNetworks.
  • 9. Copyright © 2017 Criteo RecSys’17: Contextual Extension Condition RNN on contextual information (event type, timestamp) Integrate at 2 levels: Input and Output layers Hidden Dynamics E. Smirnova and F. Vasile. Contextual Sequence Modeling for Recommendation with Recurrent Neural Networks.
  • 10. Copyright © 2017 Criteo Wrap up • Session-based recommendation has now it own track • Stronger baselines have been introduced • Multiple extensions to Recurrent Neural Networks to better model user behavior
  • 11. Copyright © 2017 Criteo Representation learning
  • 12. Copyright © 2017 Criteo Representation learning Learning to represent items, users and their relationships in an appropriate space (as a real valued vector)
  • 13. Copyright © 2017 Criteo Representation learning Leveraging available content (images, descriptions, reviews …) Helping with the cold start problem
  • 14. Copyright © 2017 Criteo Leveraging content: Review texts R. Catherine et al. Transnets: Learning to transform for recommendation • Review texts are available, how can we use them ? • Learn a representation of the review and then predict the associated rating
  • 15. Copyright © 2017 Criteo Leveraging content: Item features T. Nedelec et al. : Content2Vec: Specializing jointrepresentationsof productimagesand text for the task of product recommendation • How can we combine heterogeneous product representations ? • Specialize feature representations (text, image, …) for the task of item-item similarity and merge them
  • 16. Copyright © 2017 Criteo Cold start problem: Attribute to feature mapping • Can we use item characteristics to initialize an new item's latent representation ? • Learning attribute (item characteristics) to feature (latent space representation) mapping. D. Cohen et al. : Expediting Exploration by Attribute-to-Feature Mapping for Cold-Start Recommendation
  • 17. Copyright © 2017 Criteo Cold start problem: Attribute to feature mapping • We learn a linear mapping between attribute vectors {𝒂𝒊} and latent representation {𝒗𝒊} 𝑣𝑖 ≈ 𝑊𝑎𝑖 • For a new item 𝒋 we initialize its latent representation 𝑣𝑗 0 ≈ 𝑊𝑎𝑗 D.Cohen et al. : Expediting Exploration by Attribute-to-Feature Mapping for Cold-Start Recommendation
  • 18. Copyright © 2017 Criteo Folding • “Folding” effect of embedding can lead to spurious recommendations • Model has to take into account data Missing Not At Random, introduce metric to measure the severity of folding D. Xin et al. Folding: Why Good Models Sometimes Make Spurious Recommendations.
  • 19. Copyright © 2017 Criteo Wrap up • Embed all the things ! • A Deep MultimodalApproach for Cold-start Music Recommendation(Oramas et al.) • Comparing Neural and Attractiveness-based Visual Features for Artwork Recommendation (Dominguez et al.) • Translation based recommendation (He et al.) • Use content and reviews • InterpretableConvolutional Neural Networks with Dual Local and Global Attention for Review Rating Prediction (Seo et al.) • Recommendation of High Quality Representative Reviews in e-commerce (Paul et al.) • Sequential recommendations • Sequential User-based Recurrent Neural Network Recommendations (Donkers et al.)
  • 20. Copyright © 2017 Criteo Scalability
  • 21. Copyright © 2017 Criteo Large scale constraints Many products, many users Online latency Training time Offline storage
  • 22. Copyright © 2017 Criteo Convolution at character level for session-based reco Id Name Category Tuan et al. 3D Convolutional Networks for Session-based Recommendation with Content Features Item #1 Item #2 Item #n view view basket
  • 23. Copyright © 2017 Criteo Convolution at character level for session-based reco Id Name Category 0 a b c d … 1 2 3 4 5 6e …@? ! 1 1 1 Id: 263N Tuan et al. 3D Convolutional Networks for Session-based Recommendation with Content Features Item #1 Item #2 view view basket Item #n
  • 24. Copyright © 2017 Criteo Convolution at character level for session-based reco Id Name Category 0 a b c d … 1 2 3 4 5 6e …@? ! 1 1 1 Id: 263 Name: “iPhone” Category: “Phones/Apple” V N N x 3V Item #1 Tuan et al. 3D Convolutional Networks for Session-based Recommendation with Content Features Item #1 Item #2 view view basket Item #n
  • 25. Copyright © 2017 Criteo Convolution at character level for session-based reco Item #1 Item #2 Id Name Category 0 a b c d … 1 2 3 4 5 6e …@? ! 1 1 1 3*V = 56 N=150 Compact Input 3V x N x D Id: 263 Name: “iPhone” Category: “Phones/Apple” V N N x 3V Item #1 Tuan et al. 3D Convolutional Networks for Session-based Recommendation with Content Features view view basket Item #n
  • 26. Copyright © 2017 Criteo Convolution at character level for session-based reco Tuan et al. 3D Convolutional Networks for Session-based Recommendation with Content Features
  • 27. Copyright © 2017 Criteo Scaling deep nets Your favorite deep net D x L1 D x Ln - Compact - Fast to compute - For input and output - Reversible output - No change in the deep-net - Appropriate loss - Same accuracy Serrà et al. Getting Deep Recommenders Fit : Bloom Embeddings for Sparse Binary Input / Output Networks
  • 28. Copyright © 2017 Criteo Bloom filters embeddings 1 1 1 H1 H2 Hk 1 D m < D p Serrà et al. Getting Deep Recommenders Fit : Bloom Embeddings for Sparse Binary Input / Output Networks
  • 29. Copyright © 2017 Criteo Bloom filters embeddings 1 1 1 H1 H2 Hk 1 Your favorite deep net 0.1 0.3 0.2 D m < D m x L1 m x Ln p v Serrà et al. Getting Deep Recommenders Fit : Bloom Embeddings for Sparse Binary Input / Output Networks
  • 30. Copyright © 2017 Criteo Bloom filters embeddings 1 1 1 H1 H2 Hk 1 Your favorite deep net 0.1 0.3 0.2 D m < D m x L1 m x Ln 1 q p H1 Hk y v H2 Serrà et al. Getting Deep Recommenders Fit : Bloom Embeddings for Sparse Binary Input / Output Networks
  • 31. Copyright © 2017 Criteo Wrap up Dedicated workshop Preselection of products Scaling deep nets Online Ranking Candidates Items Offline Selection N products K products (K << N)
  • 32. Copyright © 2017 Criteo Conclusion RecSys’18 - Vancouver, Canada, 2nd-7th October 2018
  • 33. Copyright © 2017 Criteo Thank you!
  • 34. Copyright © 2017 Criteo Join Criteo!