O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
The Cloud Data Platform for Insights-Driven Enterprises
Deep Learning
An Introduction
Shivaji Dutta
(shivaji@aurius.io)
Agenda
• Introduction
• Deep Learning (DL)
• Neural Networks
• Reasons for Success
• Types of Networks and Use Cases
• DL ...
Algorithms
Categories of Learning Algorithms
• Supervised*
– Train with Labeled Dataset
• Unsupervised
– Understanding the data by fi...
Some Concepts – School Level Math Question
• Math Quiz 
– What is Equation of a Straight Line?
• Probability
– Poker Play...
Objective
• Find a function, which can predict accurately for a domain of
problem.
– We would want to find a universal fun...
Deep Learning
• It a deeply stacked layer of Neural Networks
• Neural Networks are Biologically inspired. What?
???
Deep Learning
Neural Networks
Neural Networks - Perceptron
• Series of input values {x1, x2,…,xn}
• Randomly initialized Weights {w1,w2,..wn}
• Create a...
Neural Networks – Activation Functions
Sigmoid RELU
tanh Leaky RELU
Algorithms
Neural Networks - Loss
• In Supervised learning, the idea is to compare the output
generated with the function to the Actu...
Neural Networks – Back Propagation
• Once a loss is generated
• The idea is to update the weights so that they can be chan...
“2-layer Neural Net”, or
“1-hidden-layer Neural Net”
“3-layer Neural Net”, or
“2-hidden-layer Neural Net”
“Fully-connected...
Sample Networks – Multi Layered Architecture
GoogleNet
AlexNet
ResNet
The Cloud Data Platform for Insights-Driven Enterprises
Neural Networks? So What’s
New?
ILSRVC – Imagenet
Deep Learning Disruption
Reasons for Success
• Vast amounts of Labeled Data
– E.g. ImageNet
• GPUs
– Traditionally used for Gaming
– Very Good for ...
22K categories and 14M images
www.image-net.org
Deng, Dong, Socher, Li, Li, & Fei-Fei, 2009
22 4/4/2017Lecture 1 -Fei-Fei ...
The Cloud Data Platform for Insights-Driven Enterprises
Types of Networks
Convolution Neural Network (CNN/Covnets)
• Convolution Neural Network
– Break Through Architecture started Deep Learning D...
Convolution Neural Network Architecture
Convolution Neural Network
Deep Learning: Learns increasingly complex features
pixels
edges
object parts
(combination
of edges)
object models
Recurrent Neural Networks
• Need Context
• Images do not carry context
• Languages – Complex
• ”I like spicy food, but it
...
Natural Language Processing
• Machine Translation
– Higher Accuracy
– Same Model can do multi-language translation without...
Reinforcement Learning
Deep Reinforcement Learning
Machine Learning vs Deep Learning
• Higher Accuracy Rates
– Image Recognition almost as good as humans
– Machine Translati...
The Cloud Data Platform for Insights-Driven Enterprises
Frameworks
Deep Learning Needs
• Intensive Mathematical Operations
• Working on Multidimensional Matrices
• Various Mathematical Oper...
Deep Learning Frameworks
Framework Name Adoption Organization
Tensorflow High Google
Caffe/Caffe2 Medium-High Facebook, UC...
Deep Learning Frameworks
Framework Adoption Organization
Keras High Google. Extremely popular.
Torch/PyTorch Medium - High...
Deep Learning Frameworks
Framework Adoption Organization
Neon Low-Medium Intel. Nervana acquired in
2016. Fastest DL Frame...
Language
Language Adoption
Python Very High. Most Common. Works
well with numpy, openCV, scikit-
learn.
Lua (Torch) Medium...
Frameworks
• Most Frameworks are similar and do a similar job as listed in Slide 2.
• Tensorflow and Keras are the most wi...
Other Env. topics
OS Adoption
Ubuntu (16 or 14) Very Prevalent as a default OS to
be supported
Notebooks
Jupyter Almost Al...
The Cloud Data Platform for Insights-Driven Enterprises
Hardware
Hardware
Training
• 4-8 GPU Nodes (Titanx, K80, P100, Volta*)
– On Google this would become TPU in coming months
• Multi C...
The Cloud Data Platform for Insights-Driven Enterprises
Deep Learning Ecosystem
Major Companies with AI Research
Google > 50% Mindshare of the AI Market
Competitive Landscape
Company Product Remarks
Microsoft CNTK
https://studio.azureml.net/
- A very comprehensive support fo...
Azure Machine Learning Studio
Competitive Landscape
Company Product Remarks
Alphabet Google ML Engine
Rest API Based
Vision API
Video Intelligence API
N...
Google Cloud Vision
Competitive Landscape
Company Product
H20.ai Sparkling Water and Deep Water
SigOpt Improve ML Models
DataRobot Build and D...
H20 Clarifai.ai
Sample Machine Learning – Life Cycle
Get/Prepare
Data
Build/Edit
Experiment
Create/Update
Model
Evaluate
Model
Results
Bui...
The Cloud Data Platform for Insights-Driven Enterprises
Challenges
Challenges to Deep Learning Success (Andrew Ng)
• Data. Among leading AI teams, many can likely replicate others’
software...
Modern Scientist or Stars of Deep Learning
Modern Scientist or Stars of Deep Learning
Modern Scientists
Deep Learning on Qubole Data Platform
Próximos SlideShares
Carregando em…5
×

Deep Learning on Qubole Data Platform

This is an 1 hour presentation on Neural Networks, Deep Learning, Computer Vision, Recurrent Neural Network and Reinforcement Learning. The talks later have links on how to run Neural Networks on

Deep Learning on Qubole Data Platform

  1. 1. The Cloud Data Platform for Insights-Driven Enterprises Deep Learning An Introduction Shivaji Dutta (shivaji@aurius.io)
  2. 2. Agenda • Introduction • Deep Learning (DL) • Neural Networks • Reasons for Success • Types of Networks and Use Cases • DL Frameworks Comparison • DL Ecosystem Companies
  3. 3. Algorithms
  4. 4. Categories of Learning Algorithms • Supervised* – Train with Labeled Dataset • Unsupervised – Understanding the data by finding hidden patterns • Reinforcement Learning – Learning by trial and error – Success with Games, Robotics, Simulations
  5. 5. Some Concepts – School Level Math Question • Math Quiz  – What is Equation of a Straight Line? • Probability – Poker Players  • Non Linear Function
  6. 6. Objective • Find a function, which can predict accurately for a domain of problem. – We would want to find a universal function/set of functions, but we are far away.
  7. 7. Deep Learning • It a deeply stacked layer of Neural Networks • Neural Networks are Biologically inspired. What? ???
  8. 8. Deep Learning
  9. 9. Neural Networks
  10. 10. Neural Networks - Perceptron • Series of input values {x1, x2,…,xn} • Randomly initialized Weights {w1,w2,..wn} • Create a multi layer function of functions 1. Combing Linear Functions and Non Linear (Activation) Functions 2. Stacking Each Layer of Combination on top of the other • The above is called a Perceptron
  11. 11. Neural Networks – Activation Functions Sigmoid RELU tanh Leaky RELU
  12. 12. Algorithms
  13. 13. Neural Networks - Loss • In Supervised learning, the idea is to compare the output generated with the function to the Actual Labeled Value. – E.g. if I give an image pixels as an input, I will have labeled data as output. The idea of machine learning is to minimize this loss to near 0.
  14. 14. Neural Networks – Back Propagation • Once a loss is generated • The idea is to update the weights so that they can be changed by “delta” in regards to the loss, so that the loss slowly progresses towards 0. • This is done by Differential Equation Chaining. – More details can be found on (http://neuralnetworksanddeeplearning.com/chap1.html) • Eventually after many iterations (epochs) weights and bias changes to be able to get outputs closer to actual values
  15. 15. “2-layer Neural Net”, or “1-hidden-layer Neural Net” “3-layer Neural Net”, or “2-hidden-layer Neural Net” “Fully-connected” layers Sample Neural networks: Architectures
  16. 16. Sample Networks – Multi Layered Architecture GoogleNet AlexNet ResNet
  17. 17. The Cloud Data Platform for Insights-Driven Enterprises Neural Networks? So What’s New?
  18. 18. ILSRVC – Imagenet Deep Learning Disruption
  19. 19. Reasons for Success • Vast amounts of Labeled Data – E.g. ImageNet • GPUs – Traditionally used for Gaming – Very Good for Very Large Matrix Operation – 1 Nvidia TitanX 11 Teraflops vs Xeon (500 Giga Flops) – *Google in May 2017, launched TPUs (Tensor Processing Units), custom built for TPUs • Better Activation Functions – RELU, Leaky RELU
  20. 20. 22K categories and 14M images www.image-net.org Deng, Dong, Socher, Li, Li, & Fei-Fei, 2009 22 4/4/2017Lecture 1 -Fei-Fei Li & Justin Johnson & Serena Yeung • Animals • Bird • Fish • Mammal • • Invertebrate • • Plants • • Tree • • Flower Food Materials Structures Artifact • Tools • Appliances • Structures • • • Person Scenes • Indoor • Geological Formations SportActivities
  21. 21. The Cloud Data Platform for Insights-Driven Enterprises Types of Networks
  22. 22. Convolution Neural Network (CNN/Covnets) • Convolution Neural Network – Break Through Architecture started Deep Learning Disruption. • Very Successfully in Images and Video datasets • Used in Text Classification and other use cases
  23. 23. Convolution Neural Network Architecture
  24. 24. Convolution Neural Network
  25. 25. Deep Learning: Learns increasingly complex features pixels edges object parts (combination of edges) object models
  26. 26. Recurrent Neural Networks • Need Context • Images do not carry context • Languages – Complex • ”I like spicy food, but it makes me uncomfortable”?
  27. 27. Natural Language Processing • Machine Translation – Higher Accuracy – Same Model can do multi-language translation without pre-training on every language • Question Answer System • Word Embeddings • Sentence Completions • Speech to Text (Siri, Android) • Text to Speech • Text Summarization
  28. 28. Reinforcement Learning
  29. 29. Deep Reinforcement Learning
  30. 30. Machine Learning vs Deep Learning • Higher Accuracy Rates – Image Recognition almost as good as humans – Machine Translations as good as Humans in many Western languages – Amazing accurate results with lot of traditionally hard to solve areas • No Hand Crafted Features – Traditional Machine Learning needs handcrafting of Feature Vectors – Deep Learning, No hand crafting of features
  31. 31. The Cloud Data Platform for Insights-Driven Enterprises Frameworks
  32. 32. Deep Learning Needs • Intensive Mathematical Operations • Working on Multidimensional Matrices • Various Mathematical Operations for Matrix Initialization (Gaussian, etc) • Random Value generation, Initialization functions • Support for processing on Multi-GPUs • Calculating Loss, Annealing, Decay rate • Support for Convolutions, RNN, LSTMs, GRUs • Support Multiple Gradient Descent Algorithms • Ability to persist the models • Need to be able to train fast • Visualize Losses and Accuracy
  33. 33. Deep Learning Frameworks Framework Name Adoption Organization Tensorflow High Google Caffe/Caffe2 Medium-High Facebook, UC Berkeley (Good support for Image analysis)- Caffe2 released in 2017 Mxnet Low Amazon. Released in 2017 CNTK Medium (High in Microsoft Users) Microsoft. Good example with Image Identification (COCO dataset) Theano Medium University of Montreal. One of the oldests frameworks.
  34. 34. Deep Learning Frameworks Framework Adoption Organization Keras High Google. Extremely popular. Torch/PyTorch Medium - High Open Source. Twitter uses it. Very popular in Non Python user base DeepLearning4J Medium DeepLearning4J. Small company in SF, started in 2014. Good Java and Hadoop support. Loosing grounds to Tensorflow. Chainer Low-Medium Preferred Networks. A japanese company. Applications in IOT and Robotics
  35. 35. Deep Learning Frameworks Framework Adoption Organization Neon Low-Medium Intel. Nervana acquired in 2016. Fastest DL Framework BigDL Low Intel. Support for running DeepLearning on Spark. Python Numpy like API. Built in support for Intel MKL libraries. Cloudera Supports CUDA High Nvidia. All frameworks use it and Self Driving Car industry TensorRT Low Nvidia. Optimizes the Deep Learning layers, increasing inference performance.
  36. 36. Language Language Adoption Python Very High. Most Common. Works well with numpy, openCV, scikit- learn. Lua (Torch) Medium. Used at Twitter and some universities. C++ Medium. Common with Hardware vendors and Low lever runtime implementations Java Very low. Only among Deeplearning4j users
  37. 37. Frameworks • Most Frameworks are similar and do a similar job as listed in Slide 2. • Tensorflow and Keras are the most widely adopted – Large support from Google – Support for Threading and multi GPU – First Class support for Python – Support for HDFS and Google Cloud File System – Distributed compute support
  38. 38. Other Env. topics OS Adoption Ubuntu (16 or 14) Very Prevalent as a default OS to be supported Notebooks Jupyter Almost All examples on Jupyter Notebook
  39. 39. The Cloud Data Platform for Insights-Driven Enterprises Hardware
  40. 40. Hardware Training • 4-8 GPU Nodes (Titanx, K80, P100, Volta*) – On Google this would become TPU in coming months • Multi Cluster (High Speed Network) • Training time (Days to Weeks) Inferencing • Less Compute need • CPUs (No need for GPUs) • Can run on light weight low power devices like “Smart Phones” or “pi devices”
  41. 41. The Cloud Data Platform for Insights-Driven Enterprises Deep Learning Ecosystem
  42. 42. Major Companies with AI Research Google > 50% Mindshare of the AI Market
  43. 43. Competitive Landscape Company Product Remarks Microsoft CNTK https://studio.azureml.net/ - A very comprehensive support for Machine Learning Libraries. - A well designed Interface Azure Cloud is growing very fast. They have actively taken up market share from Amazon IBM Watson Power8 PC with NVLink Historic Dominance with Deep Blue (Chess) and Jeopardy IBM BlueMix IBM uses Watson to Market itself.
  44. 44. Azure Machine Learning Studio
  45. 45. Competitive Landscape Company Product Remarks Alphabet Google ML Engine Rest API Based Vision API Video Intelligence API Natural language Translation API Deep Mind - Solving Artificial General Intelligence - Impact on Healthcare and Data Center Power Consumption Tensor Processing Unit - Competing with Nvida - Will be offered as a Cloud Service Company with largest Mindshare in Artificial Intelligence. I think Google will be the biggest competitor in the Cloud Business going forward. https://cloud.google.com/products/ Amazon - Apache Mxnet Similar Rest based API as Google Market Leader in Cloud
  46. 46. Google Cloud Vision
  47. 47. Competitive Landscape Company Product H20.ai Sparkling Water and Deep Water SigOpt Improve ML Models DataRobot Build and Deploy Machine Learning Models Clarifai.ai Image and Video Tagging Crowdflower.ai Dataset preparation for Uber and many companies
  48. 48. H20 Clarifai.ai
  49. 49. Sample Machine Learning – Life Cycle Get/Prepare Data Build/Edit Experiment Create/Update Model Evaluate Model Results Build ML Model Deploy as Web Service Provision Workspace Get Qubole Subscription Create Cluster Publish an App Integrate with App/Analytics Publish the model Deploy Model as a Web Service Examine the Predictions / Use more production data to fine tune Model
  50. 50. The Cloud Data Platform for Insights-Driven Enterprises Challenges
  51. 51. Challenges to Deep Learning Success (Andrew Ng) • Data. Among leading AI teams, many can likely replicate others’ software in, at most, 1–2 years. But it is exceedingly difficult to get access to someone else’s data. Thus data, rather than software, is the defensible barrier for many businesses. • Talent. Simply downloading and “applying” open-source software to your data won’t work. AI needs to be customized to your business context and data. This is why there is currently a war for the scarce AI talent that can do this work.
  52. 52. Modern Scientist or Stars of Deep Learning
  53. 53. Modern Scientist or Stars of Deep Learning
  54. 54. Modern Scientists

×