Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
Próximos SlideShares
Carregando em…5
×

# Collaborative DL

194 visualizações

Seminar on DL

• Full Name
Comment goes here.

Are you sure you want to Yes No

### Collaborative DL

1. 1. Review of Probabilistic Matrix Factorization Topic Modeling and Collaborative Topic Modeling Collaborative Deep Learning Review: Collaborative Deep Learning Hai D. Nguyen May 5, 2016 Hai D. Nguyen Review: Collaborative Deep Learning
2. 2. Review of Probabilistic Matrix Factorization Topic Modeling and Collaborative Topic Modeling Collaborative Deep Learning Table of contents 1 Review of Probabilistic Matrix Factorization 2 Topic Modeling and Collaborative Topic Modeling 3 Collaborative Deep Learning Hai D. Nguyen Review: Collaborative Deep Learning
3. 3. Review of Probabilistic Matrix Factorization Topic Modeling and Collaborative Topic Modeling Collaborative Deep Learning Probabilistic Matrix Factorization (PMF) T A GENERATIVE process: Pick User Factor Ui Ui ∼ N(0, σ2 u) Pick Item Factor Vj Vj ∼ N(0, σ2 v ) For each (User, Item) pair, pick Rij Rij ∼ N(UT i Vj , σ2 r ) Joint Probability: P(U, V , R) = P(U)P(V )P(R|U, V ) Graphical Model Hai D. Nguyen Review: Collaborative Deep Learning
4. 4. Review of Probabilistic Matrix Factorization Topic Modeling and Collaborative Topic Modeling Collaborative Deep Learning MAP is equivalent to Regularized Least Square MAP under gaussian distribution U∗ , V ∗ = argmax U,V P(U, V |R) = argmax U,V P(U)P(V )P(R|U, V ) = argmax U,V logP(U) + logP(V ) + logP(R|U, V ) = argmin U,V 1 2 ruv (UT V − Ruv )2 + λu 2 ||U||2 2 + λv 2 ||V ||2 2 Hai D. Nguyen Review: Collaborative Deep Learning
5. 5. Review of Probabilistic Matrix Factorization Topic Modeling and Collaborative Topic Modeling Collaborative Deep Learning Latent Dirichlet Allocation A GENERATIVE process LDA assumes the following generative process for each document w in a corpus D 1 For k = 1...K: (a) φ(k) ∼ Dirichlet(β) 2 For each document d ∈ D : (a) θd ∼ Dirichlet(α) (b) For each word wi ∈ d : i. zi ∼ Mult(θd ) ii. wi ∼ Mult(φ(zi ) ) Make use of Gibb Sampling or Variational Inference to train the model Hai D. Nguyen Review: Collaborative Deep Learning
6. 6. Review of Probabilistic Matrix Factorization Topic Modeling and Collaborative Topic Modeling Collaborative Deep Learning Collaborative Topic Models Combination of LDA and PMF In PMF, each item has a latent representation V in some unknown latent space In Topic modelling (LDA), each item (article) has topic proportions θ in the learned topic space Basic idea: Fix V = θ Hai D. Nguyen Review: Collaborative Deep Learning
7. 7. Review of Probabilistic Matrix Factorization Topic Modeling and Collaborative Topic Modeling Collaborative Deep Learning Collaborative Topic Modeling: Graphical Model and Generative Process The generative process for CTP is deﬁned as follow: Hai D. Nguyen Review: Collaborative Deep Learning
8. 8. Review of Probabilistic Matrix Factorization Topic Modeling and Collaborative Topic Modeling Collaborative Deep Learning Collaborative Deep Learning Same idea as CTP model previously mentioned but diﬀerent methods (Deep Learning) Replace Topic Model (LDA here) with a certain DEEP LEARNING model which can be DBF, SDAE or CNN. Hai D. Nguyen Review: Collaborative Deep Learning
9. 9. Review of Probabilistic Matrix Factorization Topic Modeling and Collaborative Topic Modeling Collaborative Deep Learning Collaborative Deep Learning: Graphical Model and Generative Process The generative process for CDL is deﬁned as follow: Hai D. Nguyen Review: Collaborative Deep Learning
10. 10. Review of Probabilistic Matrix Factorization Topic Modeling and Collaborative Topic Modeling Collaborative Deep Learning Collaborative Deep Learning: Learning Hai D. Nguyen Review: Collaborative Deep Learning
11. 11. Review of Probabilistic Matrix Factorization Topic Modeling and Collaborative Topic Modeling Collaborative Deep Learning Comparison between two methods: CTM vs CDL CTM method CDL method Hai D. Nguyen Review: Collaborative Deep Learning
12. 12. Review of Probabilistic Matrix Factorization Topic Modeling and Collaborative Topic Modeling Collaborative Deep Learning Some thoughs and extensions it canbe extended to multi-dimensional recommendation system (e.g., context-based RS) with Tensor approaches In my opinion, LDA can be viewed as a shalow model. In this paper, authors proposed using DL which can automatically and deeply extract features for document (features maybe more eﬀective than that of LDA) How can we extend LDA to deeper model? Hai D. Nguyen Review: Collaborative Deep Learning