O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
Confidential + Proprietary Smart Reply: Learning a Model of Conversation from Data Anjuli Kannan Software Engineer, Google...
Problem
Confidential + Proprietary Can you do Tuesday or Wednesday? Phil Sharp
Confidential + Proprietary Tuesday Wednesday Can you do Tuesday or Wednesday? Phil Sharp
Smart Reply feature ● Provide text assistance for email reply composition ● Targeted at mobile ● Responses can be sent on ...
Smart Reply feature predicts email responses Smart Reply Incoming email Response email
Why is this task hard? ● extracting meaning from previous message ● generating language ● grammatical transformations betw...
Why is this solution interesting? ● Model is learned fully from data
Model
Confidential + Proprietary Neural network Is a 4 Is a 5 ... ... Image: Wikipedia
Confidential + Proprietary Neural network Neuron Is a 4 Is a 5
Confidential + Proprietary Basic building block is the neuron Greg Corrado
Confidential + Proprietary Neural network Is a 4 Is a 5 ... ...
Confidential + Proprietary Learn a function from one space to another f(.)x ∈ Rn y ∈ Rm
Confidential + Proprietary Smartreply feature predicts email responses Smartreply Incoming email Response email
Confidential + Proprietary Recurrent neural networks handle sequences of input Diagram by Felix Gers
Confidential + Proprietary Recurrent neural networks handle sequences of input Diagram by Felix Gers
Confidential + Proprietary Recurrent neural networks handle sequences of input
Confidential + Proprietary Reading a word into a feed-forward neural network cat output
Confidential + Proprietary Reading a sequence of words into an RNN That
Confidential + Proprietary Reading a sequence of words into an RNN That is
Confidential + Proprietary Reading a sequence of words into an RNN That is good
Confidential + Proprietary Reading a sequence of words into an RNN That is good !
Confidential + Proprietary Reading a sequence of words into an RNN That is good ! output
Sequence-to-sequence model Sutskever et al, NIPS 2014
Sequence-to-sequence model encoder decoder
Sequence-to-sequence model Ingests incoming message Generates reply message
Inference
Reading a sequence of words into an RNN How
Reading a sequence of words into an RNN How are
Reading a sequence of words into an RNN How are you
Reading a sequence of words into an RNN How are you ?
Encoder ingests the incoming message How are you ? Internal state is a fixed length encoding of the message
Decoder is initialized with final state of encoder How are you ? __
Decoder is initialized with final state of encoder How are you ? __
Decoder predicts next word How are you ? __
Decoder predicts next word How are you ? ____ I
Smartreply model How Message
Smartreply model How are Message
Smartreply model How are you Message
Smartreply model How are you ? Message
Smartreply model How are you ? __ I Message Response
Smartreply model How are you ? __ I I am Message Response
Smartreply model How are you ? __ I am I am great Message Response
Smartreply model How are you ? __ I am great I am great ! Message Response Vinyals & Le, ICML DL 2015
Inference ● Resulting model is fully generative ● Output distribution can be used to determine the most likely responses u...
Training
Training ● Training data is a corpus of email-reply pairs ● Both encoder and decoder are trained together (end-to-end)
Training ● Training data is a corpus of email-reply pairs ● Both encoder and decoder are trained together (end-to-end)
Confidential + Proprietary Key points about model ● Everything is learned from data, even features ● Neural network smooth...
Smart Reply in Production
Deployment & coverage ● Deployed in Inbox by Gmail ● Used to assist with more than 10% of all mobile replies
Examples
Quality ● How do we ensure that the response options are always high quality in content and language? ○ Avoid incorrect gr...
Most frequently used clusters
Confidential + Proprietary What the model can do
Confidential + Proprietary What the model can't do ● Match every user's tone and style
Confidential + Proprietary What the model can't do ● Match every user's tone and style ● Ensure diverse options
Confidential + Proprietary What the model can't do ● Match every user's tone and style ● Ensure diverse options ● Access a...
Conclusions
Conclusions ● Sequence-to-sequence produces plausible email replies in many common scenarios, when trained on an email cor...
Confidential + Proprietary Conclusions ● A conversation model learned entirely from data is very powerful ● A data-driven ...
Confidential + Proprietary Collaborators - Greg Corrado, Oriol Vinyals (Google Brain) - Balint Miklos, Tobias Kaufman, Las...
Confidential + Proprietary Thank you!
Extra slides
Example
Unique cluster and suggestion usage
Ranking experiments
Próximos SlideShares
Carregando em…5
×

Anjuli Kannan, Software Engineer, Google at MLconf SF 2016

1.329 visualizações

Publicada em

Smart Reply: Learning a Model of Conversation from Data: Smart Reply is a text assistance feature that was recently introduced to Inbox by Gmail. Given an incoming email message, the Smartreply system analyzes its contents and suggests complete responses that the recipient can send with just one tap. This talk will cover how we built Smartreply using a combination of deep learning and semantic clustering, as well as what we learned along the way and why we think it shows promise for the future of dialogue models.

Publicada em: Tecnologia
no profile picture user

  • Entre para ver os comentários

Anjuli Kannan, Software Engineer, Google at MLconf SF 2016

  1. 1. Confidential + Proprietary Smart Reply: Learning a Model of Conversation from Data Anjuli Kannan Software Engineer, Google Brain
  2. 2. Problem
  3. 3. Confidential + Proprietary Can you do Tuesday or Wednesday? Phil Sharp
  4. 4. Confidential + Proprietary Tuesday Wednesday Can you do Tuesday or Wednesday? Phil Sharp
  5. 5. Smart Reply feature ● Provide text assistance for email reply composition ● Targeted at mobile ● Responses can be sent on their own or extended
  6. 6. Smart Reply feature predicts email responses Smart Reply Incoming email Response email
  7. 7. Why is this task hard? ● extracting meaning from previous message ● generating language ● grammatical transformations between call and response ● matching style/tone
  8. 8. Why is this solution interesting? ● Model is learned fully from data
  9. 9. Model
  10. 10. Confidential + Proprietary Neural network Is a 4 Is a 5 ... ... Image: Wikipedia
  11. 11. Confidential + Proprietary Neural network Neuron Is a 4 Is a 5
  12. 12. Confidential + Proprietary Basic building block is the neuron Greg Corrado
  13. 13. Confidential + Proprietary Neural network Is a 4 Is a 5 ... ...
  14. 14. Confidential + Proprietary Learn a function from one space to another f(.)x ∈ Rn y ∈ Rm
  15. 15. Confidential + Proprietary Smartreply feature predicts email responses Smartreply Incoming email Response email
  16. 16. Confidential + Proprietary Recurrent neural networks handle sequences of input Diagram by Felix Gers
  17. 17. Confidential + Proprietary Recurrent neural networks handle sequences of input Diagram by Felix Gers
  18. 18. Confidential + Proprietary Recurrent neural networks handle sequences of input
  19. 19. Confidential + Proprietary Reading a word into a feed-forward neural network cat output
  20. 20. Confidential + Proprietary Reading a sequence of words into an RNN That
  21. 21. Confidential + Proprietary Reading a sequence of words into an RNN That is
  22. 22. Confidential + Proprietary Reading a sequence of words into an RNN That is good
  23. 23. Confidential + Proprietary Reading a sequence of words into an RNN That is good !
  24. 24. Confidential + Proprietary Reading a sequence of words into an RNN That is good ! output
  25. 25. Sequence-to-sequence model Sutskever et al, NIPS 2014
  26. 26. Sequence-to-sequence model encoder decoder
  27. 27. Sequence-to-sequence model Ingests incoming message Generates reply message
  28. 28. Inference
  29. 29. Reading a sequence of words into an RNN How
  30. 30. Reading a sequence of words into an RNN How are
  31. 31. Reading a sequence of words into an RNN How are you
  32. 32. Reading a sequence of words into an RNN How are you ?
  33. 33. Encoder ingests the incoming message How are you ? Internal state is a fixed length encoding of the message
  34. 34. Decoder is initialized with final state of encoder How are you ? __
  35. 35. Decoder is initialized with final state of encoder How are you ? __
  36. 36. Decoder predicts next word How are you ? __
  37. 37. Decoder predicts next word How are you ? ____ I
  38. 38. Smartreply model How Message
  39. 39. Smartreply model How are Message
  40. 40. Smartreply model How are you Message
  41. 41. Smartreply model How are you ? Message
  42. 42. Smartreply model How are you ? __ I Message Response
  43. 43. Smartreply model How are you ? __ I I am Message Response
  44. 44. Smartreply model How are you ? __ I am I am great Message Response
  45. 45. Smartreply model How are you ? __ I am great I am great ! Message Response Vinyals & Le, ICML DL 2015
  46. 46. Inference ● Resulting model is fully generative ● Output distribution can be used to determine the most likely responses using a beam search
  47. 47. Training
  48. 48. Training ● Training data is a corpus of email-reply pairs ● Both encoder and decoder are trained together (end-to-end)
  49. 49. Training ● Training data is a corpus of email-reply pairs ● Both encoder and decoder are trained together (end-to-end)
  50. 50. Confidential + Proprietary Key points about model ● Everything is learned from data, even features ● Neural network smooths across language variation
  51. 51. Smart Reply in Production
  52. 52. Deployment & coverage ● Deployed in Inbox by Gmail ● Used to assist with more than 10% of all mobile replies
  53. 53. Examples
  54. 54. Quality ● How do we ensure that the response options are always high quality in content and language? ○ Avoid incorrect grammar and mechanics, misspellings e.g., your the best ○ Avoid inappropriate, offensive responses. e.g., Leave me alone. ○ Deal with wide variability, informal language. e.g., got it thx ● Restricting model vocabulary is not sufficient! Solution: Restrict to a fixed set of valid responses, derived automatically from data.
  55. 55. Most frequently used clusters
  56. 56. Confidential + Proprietary What the model can do
  57. 57. Confidential + Proprietary What the model can't do ● Match every user's tone and style
  58. 58. Confidential + Proprietary What the model can't do ● Match every user's tone and style ● Ensure diverse options
  59. 59. Confidential + Proprietary What the model can't do ● Match every user's tone and style ● Ensure diverse options ● Access and update any kind of state or knowledge base
  60. 60. Conclusions
  61. 61. Conclusions ● Sequence-to-sequence produces plausible email replies in many common scenarios, when trained on an email corpus ● Smart Reply is deployed in Inbox by Gmail and generates more than 10% of mobile replies
  62. 62. Confidential + Proprietary Conclusions ● A conversation model learned entirely from data is very powerful ● A data-driven approach can be complementary to hand-crafted rules and scenarios
  63. 63. Confidential + Proprietary Collaborators - Greg Corrado, Oriol Vinyals (Google Brain) - Balint Miklos, Tobias Kaufman, Laszlo Lukacs, and Karol Kurach (GMail) - Sujith Ravi (Google Research)
  64. 64. Confidential + Proprietary Thank you!
  65. 65. Extra slides
  66. 66. Example
  67. 67. Unique cluster and suggestion usage
  68. 68. Ranking experiments

×