O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Location Embeddings for Next Trip Recommendation

174 visualizações

Publicada em

Joint work wih Amadeus presenting a recommender system for your next destination using knowledge graphs and deep learning network

Publicada em: Tecnologia
  • Seja o primeiro a comentar

  • Seja a primeira pessoa a gostar disto

Location Embeddings for Next Trip Recommendation

  1. 1. Location Embeddings for Next Trip Recommendation Amine Dadoun, Raphael Troncy, Riccardo Petitti, Olivier Ratier LocWeb19, 13 May 2019
  2. 2. LocWeb 2019 … Why? LocationUser Web, Social Media Recommendation, Travel 2
  3. 3. Travel … A great source of inspiration John Doe “I do not know where to go” “Try this” 3
  4. 4. Use Case Description Given a traveler, his demographics, his historical bookings and the contextual data related to these bookings, we recommend him a ranked list of destinations he would like to go to. Traveler's Demographic Data 43 years old, Malaysian, Male, Nature, Museums Time Contextual Data 14/09/2016, Wednesday, 2 Days, Alone, etc. 21/12/2016, Friday, 14 Days, 4 persons in party, etc. 07/06/2017, Saturday, 10 Days, 2 persons in party, etc. 15/01/2017, Sunday, 5 Days, Alone, etc. 09/09/2018, Sunday, 4 Days, Alone, etc. ? + 4
  5. 5. Scientific Problems Given historical purchases made by a user (or user-item past interactions), plus the context where the interaction was made, how can we accurately predict what will be the next item the user is going to interact with? Research Questions 1. What item to recommend to the user? 2. Can we integrate external data to improve the accuracy of a predictive model? 3. How can we evaluate the recommendation made to this user? 5
  6. 6. DKFM (our approach): It combines Factorization Machines in order to represent contextual information and the WDL Recommender System in order to have the user- item interactions and the content information. The combination of these two models are represented in a DNN 6 State of the Art Recommender System Collaborative Filtering [1, 2, 3] Implicit MF Bayesian Personalized MF Neural Collaborative Filtering Content-based Filtering [4] Item KNN Hybrid Method [5] Wide & Deep Learning Context-aware Recommender System [6, 7] Factorization Machines Neural Factorization Machines Knowledge-aware Recommender System [8] Deep Knowledge Factorization Machines Collaborative Fileting: They are Matrix Factorization methods based only on the user- item interaction. They vary either on the loss used in the training or in the interaction function that computes the recommendation probability. Content-based Filtering: Item KNN is a neighborhood based collaborative filtering method, it computes the k nearest neighbors for each item. Hybrid Method: WDL is a DNN Model that computes the probability to have a user-item pair based on both user-item interaction and the content of the item Context-aware Recommender System: These two methods are based on factorization machines algorithm which take into account the context of the recommendation in addition to the user-item interaction Our ModelSota & baselines Recommender Systems
  7. 7. 7 Data integration to enrich the representation of destination User Items 𝑢𝑢1 𝑖𝑖1 𝑖𝑖2 𝑖𝑖3 ... User-Item Interactions Age, Nationality, Gender, Etc. User’s Demographics Date, Session behavior, Etc. Interaction Information Item description: • Text • Knowledge Graph • Etc. Content Information
  8. 8. 8 Contribution: Deep Knowledge Factorization Machines (DKFM) Deep Neural Network: • Collaborative information • Content information • Contextual information User Items 𝑢𝑢1 𝑖𝑖1 𝑖𝑖2 𝑖𝑖3 ... User-Item Interactions Item Description: • Text • Knowledge Graph • Etc. Content Information Age, Nationality, Gender, Etc. User’s Demographics Date, Session Behavior, Etc. Interaction Information
  9. 9. 9 Back to our problem … Next Trip Destination Traveler's Demographic Data 43 years old, Malaysian, Male, Nature, Museums 14/09/2016 Wednesday 2 Days Alone 21/12/2016 Friday 14 Days 4 persons in party 07/06/2017 Saturday 10 Days 2 persons in party 09/09/2018 Sunday 4 Days Alone ? Historical Bookings with contextual information Next Trip Recommendation
  10. 10. 10 Traveller's Profiles Data • Traveler’s Data from: • Number of Profiles: ~20M • Number of Trips: ~15 M• Trip Type: One-way, Round-Trip, Multiple Journeys Trip • Time range: February 2013- October 2019 • Number of Destinations: 1146 • Booking Creation Date • Stay Duration • Origin Airport • Origin City • Origin Country • Origin Region • Destination Airport • Destination City • Destination Country • Destination Region • Departure Date • Departure Day of the Week • Arrival Date • Advanced Purchase • Advanced Check-in • Trip Number in Party TripCustomer • Age • Customer Value • Days to Next Bday • Days to Next Flight • Nationality • Gender • Last Booking Date • Last Flown Date • Type of Services • Service Code Trip Services Traveller
  11. 11. Data Pre-processing Pipeline • Trips • Traveler demographics Remove Travelers with less than 5 Trips • Remove Travelers with less than 5 different Trips • Remove Destinations visited less than 20 times Only 32% of the trips left Only 4% of the trips left Business Leisure Only 2% of the trips left Number of Travelers 26K/20M (0.13%) Number of Trips 300K/15M (2.1%) Number of Destinations 119/1146 (10%) Travelers Segmentation 11
  12. 12. 12 Data Pre-processing: Data Filtering for Recommendation • Remove Travelers with less than 5 Trips (Different Destinations) • Remove Destinations that are visited less than 20 Times Kuala Lumpur Sydney London New York Paris Traveler 1 8 2 1 0 0 Traveler 2 4 0 1 0 1 Traveler 3 2 2 2 1 0 Traveler 4 4 0 0 0 2 Traveler 5 1 0 2 0 3 • Number of Trips: ~4.8 M bookings • Number of Travelers: 814 919 • Number of Destinations: 763 R = • Sparsity is defined as follows: 𝜌𝜌 𝑅𝑅 = 1 − #𝐼𝐼 𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼 𝐼𝐼𝐼𝐼 #𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈 × #𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼𝐼 #Feedbacks #Interactions #Cities #Travelers Sparsity 610 515 361 412 135 31 205 92% • 𝜌𝜌(Leisure_Trips) = 99.8%: Too sparse to build a Recommender System • More than 65% of travelers have traveled only 2 times • Interaction Matrix: 𝑅𝑅 ∈ 𝑁𝑁#𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 × #𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷 : 𝑟𝑟𝑢𝑢𝑢𝑢 = #𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑡𝑡𝑡𝑡 𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷 𝑖𝑖 𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑢𝑢
  13. 13. 13 Data Pre-processing: Customer Segmentation CEM Trips Business Leisure Historical Trips already labeled B/L Training B/L Classifier Prediction Trips Data: 122 242 trips Features used: • Number of Passenger, Stay Duration, Saturday Stay, Purchase Anticipation, Age, Gender Time Range: • Feb 2014 - Feb 2017 Distribution: • 40-60 % B/L Training Random Forest Classifier Grid Search on Training Data 5 Fold Cross Validation for evaluation with 75-25% Training & Test Set Accuracy = 0.87, Precision = 0.87, Recall = 0.91 Features Importance #Feedbacks #Interactions #Cities #Travelers Sparsity 304 019 152 547 119 26 019 95%
  14. 14. 14 Data Enrichment using Word Embeddings Phuket Adelaide London Etc. Cities … Wikipedia Cities Content 1. Compute the TF-IDF of each word the a Etc. Pre-trained Word Vectors [8] 2. London Textual Embedding: Weighted sum of word vectors, where the weight of each word vector corresponds to the term frequency-inverse document frequency (TF-IDF) of the word
  15. 15. 15 Data Enrichment using Knowledge Graph Embeddings Knowledge Graph Embeddings (KGE) Phuket Adelaide London Etc. Cities TransE Model[9] : Given a triple (h, r, t) in the graph, the idea is to minimize the distance between h and t embeddings KGE_Phuket KGE_Adelaide KGE_London Etc. KGE Cities Knowledge Graph Embedding of Phuket Semantic Trails Knowledge Graph: The knowledge graph represents the interaction user-venue, through the property ’visiting’ as well as the relations between the venue and the other entities, namely: category, schema and city https://arxiv.org/abs/1812.04367
  16. 16. 16 Deep Knowledge Factorization Machines Deep Neural Network: • Collaborative information • Content information • Contextual information Semantic Trails Knowledge Graph • What characterized a city the most? • An Embedding of each city is constructed based on TransE model • TransE Model: Given a triple (h, r, t) in the graph, the idea is to minimize the distance between h and t embeddings Wikipedia • Representation of cities based on their textual description in Wikipedia • Each Wikipedia Document is encoded as a weighted sum of word vectors • We used pre-trained word vectors from fasttext (n-gram model) • N-gram model is similar to Skip-gram model, but instead of learning a vector representation for a word, we learn a representation for each character. • Weights of the word vectors are their TF-IDF scores Travelers' Profiles & Trips External Data
  17. 17. Leave-one-out protocol: for each user, we remove the last destination he went to, and consider it as test set 17 Training Procedure and Evaluation ProtocolTime Training Data Test Data Recommender System Non Existing Traveler-Destination pair Recommender System trained Ranked list of Destinations Prediction … 1. 2. 4. 3.  Hitrate@K [3]  MRR@K [7] Adelaide Osaka Phuket Brunei
  18. 18. 18 Results: DKFM vs Baselines Our Model
  19. 19. 19 DKFM: what is the contribution of each input data? Better Deep Neural Network + Data Enrichment => Best results Demographics Data Textual Embedding Knowledge Graph Embedding HR@ 10 MRR@ 10 0.72 0.34 0.79 0.37 0.80 0.38 0.82 0.38 0.84 0.41 0.85 0.42 0.88 0.44 Input Contribution
  20. 20. 20 Conclusion and Future Work Future Work • Enrich cities’ characteristics using visual embeddings • Explore other loss functions such as pairwise loss • Explore the use of similarity measure inside the DNN such as cosine similarity Conclusions • Combining different types of input improves remarkably recommendation results • DKFM model outperforms state-of-the-art collaborative filtering methods Open Science • DKFM implementation available at https://gitlab.eurecom.fr/amadeus/DKFM-recommendation
  21. 21. 21 References [1] Badrul Sarwar, George Karypis, Joseph A Konstan, and John Riedl. 2001. Item-based collaborative filtering recommendation algorithms. [2] Y. Hu, Y. Koren, and C. Volinsky. 2008. Collaborative Filtering for Implicit Feedback Datasets. [3] Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. [4] Steffen Rendle. 2010. Factorization Machines. [5] Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra,Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, RohanAnil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, and Hemal Shah.2016. Wide & Deep Learning for Recommender Systems. [6] Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural Collaborative Filtering. [7] Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: A Factorization-Machine based Neural Network for CTR Prediction. [8] Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin. 2018. Advances in Pre-Training Distributed Word Representations. [9] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating Embeddings for Modeling Multi-relational Data.