O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Machine Learning Applied - Contextual Chatbots Coding, Oracle JET and TensorFlow

Oracle Code One 2018 San Francisco session

  • Entre para ver os comentários

  • Seja a primeira pessoa a gostar disto

Machine Learning Applied - Contextual Chatbots Coding, Oracle JET and TensorFlow

  1. 1. Machine Learning Applied - Contextual Chatbots Coding, Oracle JET andTensorFlow Andrejus Baranovskis, CEO andTechnical Expert, Red Samurai Consulting Oracle ACE Director and Oracle Groundbreaker Ambassador
  2. 2. Oracle ExpertsTeam ADF, JET, ORACLE FUSION, ORACLE CLOUD, MACHINE LEARNING Oracle PaaS Partner Community Award for Outstanding Java Cloud Service Contribution 2017
  3. 3. Session Goal HowTo BuildYour Own Machine Learning Chatbot
  4. 4. AGENDA • Technical Architecture • Solution WalkThrough • Machine Learning Introduction • Implementation Points
  5. 5. TECHNICAL ARCHITECTURE
  6. 6. Machine Learning Chatbot Context Communication Chatbot UI Classification Chatbot messaging Oracle JET
  7. 7. Chatbot Custom application logic Generic listener Oracle JET
  8. 8. CHATBOT CONTEXT • Chatbot framework needs a structure in which conversational intents are defined (this can be JSON file) • Conversational intent contains: • tag (unique name) • patterns (sentence patterns for neural network text classifier) • responses (one will be used as a response)
  9. 9. SOLUTION WALKTHROUGH
  10. 10. GENTLE INTRODUCTIONTO MACHINE LEARNING
  11. 11. LEARNING AND INFERENCE Training data Feature vector Learning algorithm Model Test data Feature vector Model Prediction
  12. 12. KEY PARAMETERS • Cost Function - score for each candidate parameter, shows sum of errors in predicting.The higher the cost, the worse the model parameters will be • Epoch - each step of looping through all data to update the model parameters • Learning rate - the size of the learning step
  13. 13. REGRESSION Regression algorithm Input Output Continuous Continuous Discrete
  14. 14. REGRESSION EXAMPLE w - parameter to be found usingTensorFlow
  15. 15. CLASSIFICATION f{x} Input Output DiscreteContinuous Discrete Classifier
  16. 16. CLASSIFICATION EXAMPLE Linear boundary line learned from the training data - equal probability for both groups
  17. 17. WHYTENSORFLOW? • TensorFlow has become the tool of choice to implement machine learning solutions • Developed by Google and supported by its flourishing community • Gives a way to easily implement industry-standard code
  18. 18. CLASSIFICATION IMPLEMENTATION
  19. 19. STEP 1: PREPARING DATA • Tokenise patterns into array of words • Lower case and stem all words. Example: Pharmacy => pharm. Attempt to represent related words • Create list of classes - intents • Create list of documents - combination between list of patterns and list of intents
  20. 20. STEP 1: PREPARING DATA
  21. 21. STEP 2: PREPARINGTENSORFLOW INPUT • [X: [0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, ...N],Y: [0, 0, 1, 0, 0, 0, ...M]] • [X: [0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, ...N],Y: [0, 0, 0, 1, 0, 0, ...M]] • Array representing pattern with 0/1. N = vocabulary size. 1 when word position in vocabulary is matching word from pattern • Array representing intent with 0/1. M = number of intents. 1 when intent position in list of intents/classes is matching current intent
  22. 22. STEP 2: PREPARINGTENSORFLOW INPUT
  23. 23. STEP 3:TRAINING NEURAL NETWORK • Use tflearn - deep learning library featuring a higher-level API forTensorFlow • Define X input shape - equal to word vocabulary size • Define two layers with 8 hidden neurones - optimal for text classification task (based on experiments) • DefineY input shape - equal to number of intents • Apply regression to find the best equation parameters
  24. 24. STEP 3:TRAINING NEURAL NETWORK • Define Deep Neural Network model (DNN) • Run model.fit to construct classification model. Provide X/Y inputs, number of epochs and batch size • Per each epoch, multiple operations are executed to find optimal model parameters to classify future input converted to array of 0/1
  25. 25. STEP 3:TRAINING NEURAL NETWORK • Batch size: • Smaller batch size requires less memory. Especially important for datasets with large vocabulary • Typically networks train faster with smaller batches.Weights and network parameters are updated after each propagation • The smaller the batch the less accurate estimate of the gradient (function which describes the data) could be
  26. 26. STEP 3:TRAINING NEURAL NETWORK
  27. 27. STEP 4: INITIAL MODELTESTING • Tokenise input sentence - split it into array of words • Create bag of words (array with 0/1) for the input sentence - array equal to the size of vocabulary, with 1 for each word found in input sentence • Run model.predict with given bag of words array, this will return probability for each intent
  28. 28. STEP 4: INITIAL MODELTESTING
  29. 29. STEP 5: REUSETRAINED MODEL • For better reusability, it is recommended to create separate TensorFlow notebook, to handle classification requests • We can reuse previously created DNN model, by loading it with TensorFlow pickle
  30. 30. STEP 5: REUSETRAINED MODEL
  31. 31. STEP 6:TEXT CLASSIFICATION • Define REST interface, so that function will be accessible outside TensorFlow • Convert incoming sentence into bag of words array and run model.predict • Consider results with probability higher than 0.25 to filter noise • Return multiple identified intents (if any), together with assigned probability
  32. 32. STEP 6:TEXT CLASSIFICATION
  33. 33. IMPLEMENTATION POINTS
  34. 34. QUESTIONS
  35. 35. CONTACTS • Andrejus Baranovskis (https://andrejusb.blogspot.com) • Email: abaranovskis@redsamuraiconsulting.com • Twitter: @andrejusb • LinkedIn: https://www.linkedin.com/in/andrejus-baranovskis-251b392 • Web: http://redsamuraiconsulting.com
  36. 36. REFERENCES • Source Code - https://github.com/abaranovskis-redsamurai/shenzhen • Contextual Chatbot inTensorFlow - https://bit.ly/2pFbTw4 • TensorFlow Book - http://tensorflowbook.com/

×