O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Introduction to Deep Learning with TensorFlow

25.879 visualizações

Publicada em

A practical guide for using TensorFlow with examples

Publicada em: Engenharia

Introduction to Deep Learning with TensorFlow

  1. 1. Terry Taewoong Um (terry.t.um@gmail.com) University of Waterloo Department of Electrical & Computer Engineering Terry Taewoong Um INTRODUCTION TO DEEP NEURAL NETWORK WITH TENSORFLOW 1
  2. 2. Terry Taewoong Um (terry.t.um@gmail.com) CONTENTS 2 1. Why Deep Neural Network
  3. 3. Terry Taewoong Um (terry.t.um@gmail.com) 3 EXAMPLE CASE - Imagine you have extracted features from sensors - The dimension of each sample (which represents one of gestures) is around 800 - You have 70,000 samples (trial) - What method would you apply?
  4. 4. Terry Taewoong Um (terry.t.um@gmail.com) 4 EXAMPLE CASE - Reduce the dimension from 800 to 40 by using a feature selection or dim. reduction technique ☞ What you did here is “Finding a good representation” - Then, you may apply a classification methods to classify 10 classes • You may have several ways to do it • But, what if - You have no idea for feature selection? - The dimension is much higher than 800 and you have more classes. ?
  5. 5. Terry Taewoong Um (terry.t.um@gmail.com) 5 EXAMPLE CASE - Reduce the dimension from 800 to 40 by using a feature selection or dim. reduction technique ☞ What you did here is “Finding a good representation” - Then, you may apply a classification methods to classify 10 classes • You may have several ways to do it • But, what if - You have no idea for feature selection? - The dimension is much higher than 800 and you have more classes. MNIST dataset (65000spls * 784dim) MNIST dataset (60000spls * 1024dim)
  6. 6. Terry Taewoong Um (terry.t.um@gmail.com) 6 CLASSIFICATION RESULTS error rate : 28% → 15% → 8% (2010) (2014)(2012) http://rodrigob.github.io/are_we_there_yet/bu ild/classification_datasets_results.html
  7. 7. Terry Taewoong Um (terry.t.um@gmail.com) 7 PARADIGM CHANGE Knowledge PRESENT Representation (Features) How can we find a good representation? IMAGE SPEECH Hand-Crafted Features
  8. 8. Terry Taewoong Um (terry.t.um@gmail.com) 8 PARADIGM CHANGE IMAGE SPEECH Hand-Crafted Features Knowledge PRESENT Representation (Features) Can we learn a good representation (feature) for the target task as well?
  9. 9. Terry Taewoong Um (terry.t.um@gmail.com) 9 UNSUPERVISED LEARNING “Convolutional deep belief networks for scalable unsupervised learning of hierarchical representation”, Lee et al., 2012
  10. 10. Terry Taewoong Um (terry.t.um@gmail.com) 10 THREE TYPES OF DEEP LEARNING • Unsupervised learning method Autoencoder http://goo.gl/s6kmqY - Restricted Boltzmann Machine(RBM), Autoencoder, etc. - It helps to avoid local minima problem (It regularizes the training data) - But it is not necessary when we have large amount of data. (Drop-out is enough for regularization) • Convolutional Neural Network (ConvNet) • Recurrent Neural Network (RNN) + Long-Short Term Memory (LSTM) - ConvNet has shown outstanding performance in recognition tasks (image, speech) - ConvNet contains hierarchical abstraction process called pooling. - RNN+LSTM makes use of long-term memory → Good for time-series data - RNN is a generative model: It can generate new data
  11. 11. Terry Taewoong Um (terry.t.um@gmail.com) CONTENTS 11 2. DNN with TensorFlow Thanks to Sungjoon Choi https://github.com/sjchoi86/
  12. 12. Terry Taewoong Um (terry.t.um@gmail.com) 12 DEEP LEARNING LIBRARIES
  13. 13. Terry Taewoong Um (terry.t.um@gmail.com) 13 DEEP LEARNING LIBRARY
  14. 14. Terry Taewoong Um (terry.t.um@gmail.com) 14 DEEP LEARNING LIBRARY • Karpathy’s Recommendation
  15. 15. Terry Taewoong Um (terry.t.um@gmail.com) 15 BASIC WORKFLOW OF TF 1. Load data 2. Define the NN structure 3. Set optimization parameters 4. Run! https://github.com/terryum/TensorFlow_Exercises
  16. 16. Terry Taewoong Um (terry.t.um@gmail.com) 16 EXAMPLE 1 https://github.com/terryum/TensorFlow_Exercises
  17. 17. Terry Taewoong Um (terry.t.um@gmail.com) 17 1. LOAD DATA https://github.com/terryum/TensorFlow_Exercises/blob/ master/2_LogisticRegression_MNIST_160516.ipynb
  18. 18. Terry Taewoong Um (terry.t.um@gmail.com) 18 1. LOAD DATA
  19. 19. Terry Taewoong Um (terry.t.um@gmail.com) 19 2. DEFINE THE NN STRUCTURE 3. SET OPTIMIZATION PARAMETERS
  20. 20. Terry Taewoong Um (terry.t.um@gmail.com) 20 4. RUN
  21. 21. Terry Taewoong Um (terry.t.um@gmail.com) 21 4. RUN (C.F.)
  22. 22. Terry Taewoong Um (terry.t.um@gmail.com) 22 EXAMPLE 2 https://github.com/terryum/TensorFlow_Exercises
  23. 23. Terry Taewoong Um (terry.t.um@gmail.com) 23 NEURAL NETWORK Hugo Larochelle, http://www.dmi.usherb.ca/~larocheh/index_en.html • Activation functions http://goo.gl/qMQk5H • Basic NN structure
  24. 24. Terry Taewoong Um (terry.t.um@gmail.com) 24 1. LOAD DATA https://github.com/terryum/TensorFlow_Exercises/blob/ master/3a_MLP_MNIST_160516.ipynb
  25. 25. Terry Taewoong Um (terry.t.um@gmail.com) 25 2. DEFINE THE NN STRUCTURE
  26. 26. Terry Taewoong Um (terry.t.um@gmail.com) 26 3. SET OPTIMIZATION PARAMETERS
  27. 27. Terry Taewoong Um (terry.t.um@gmail.com) 27 4. RUN
  28. 28. Terry Taewoong Um (terry.t.um@gmail.com) 28 EXAMPLE 3 https://github.com/terryum/TensorFlow_Exercises
  29. 29. Terry Taewoong Um (terry.t.um@gmail.com) 29 CONVOLUTION http://colah.github.io/posts/2014-07- Understanding-Convolutions/ http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/
  30. 30. Terry Taewoong Um (terry.t.um@gmail.com) 30 CONVOLUTIONAL NN • How can we deal with real images which is much bigger than MNIST digit images? - Use not fully-connected, but locally-connected NN - Use convolutions to get various feature maps - Abstract the results into higher layer by using pooling - Fine tune with fully-connected NN https://goo.gl/G7kBjI https://goo.gl/Xswsbd http://goo.gl/5OR5oH
  31. 31. Terry Taewoong Um (terry.t.um@gmail.com) 31 1. LOAD DATA https://github.com/terryum/TensorFlow_Exercises/blob/ master/4a_CNN_MNIST_160517.ipynb
  32. 32. Terry Taewoong Um (terry.t.um@gmail.com) 32 2. DEFINE THE NN STRUCTURE
  33. 33. Terry Taewoong Um (terry.t.um@gmail.com) 33 2. DEFINE THE NN STRUCTURE
  34. 34. Terry Taewoong Um (terry.t.um@gmail.com) 34 3. SET OPTIMIZATION PARAMETERS
  35. 35. Terry Taewoong Um (terry.t.um@gmail.com) 35 4. RUN
  36. 36. Terry Taewoong Um (terry.t.um@gmail.com) 36 4. RUN (C.F.)
  37. 37. Terry Taewoong Um (terry.t.um@gmail.com) 37 Thank you https://www.facebook.com/terryum http://terryum.io/ http://t-robotics.blogspot.kr/

×