O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
Próximos SlideShares
Carregando em…5
×

# Deep Learning on Qubole Data Platform

This is an 1 hour presentation on Neural Networks, Deep Learning, Computer Vision, Recurrent Neural Network and Reinforcement Learning. The talks later have links on how to run Neural Networks on

• Full Name
Comment goes here.

Are you sure you want to Yes No
Your message goes here
• Seja o primeiro a comentar

### Deep Learning on Qubole Data Platform

1. 1. The Cloud Data Platform for Insights-Driven Enterprises Deep Learning An Introduction Shivaji Dutta (shivaji@aurius.io)
2. 2. Agenda • Introduction • Deep Learning (DL) • Neural Networks • Reasons for Success • Types of Networks and Use Cases • DL Frameworks Comparison • DL Ecosystem Companies
3. 3. Algorithms
4. 4. Categories of Learning Algorithms • Supervised* – Train with Labeled Dataset • Unsupervised – Understanding the data by finding hidden patterns • Reinforcement Learning – Learning by trial and error – Success with Games, Robotics, Simulations
5. 5. Some Concepts – School Level Math Question • Math Quiz  – What is Equation of a Straight Line? • Probability – Poker Players  • Non Linear Function
6. 6. Objective • Find a function, which can predict accurately for a domain of problem. – We would want to find a universal function/set of functions, but we are far away.
7. 7. Deep Learning • It a deeply stacked layer of Neural Networks • Neural Networks are Biologically inspired. What? ???
8. 8. Deep Learning
9. 9. Neural Networks
10. 10. Neural Networks - Perceptron • Series of input values {x1, x2,…,xn} • Randomly initialized Weights {w1,w2,..wn} • Create a multi layer function of functions 1. Combing Linear Functions and Non Linear (Activation) Functions 2. Stacking Each Layer of Combination on top of the other • The above is called a Perceptron
11. 11. Neural Networks – Activation Functions Sigmoid RELU tanh Leaky RELU
12. 12. Algorithms
13. 13. Neural Networks - Loss • In Supervised learning, the idea is to compare the output generated with the function to the Actual Labeled Value. – E.g. if I give an image pixels as an input, I will have labeled data as output. The idea of machine learning is to minimize this loss to near 0.
14. 14. Neural Networks – Back Propagation • Once a loss is generated • The idea is to update the weights so that they can be changed by “delta” in regards to the loss, so that the loss slowly progresses towards 0. • This is done by Differential Equation Chaining. – More details can be found on (http://neuralnetworksanddeeplearning.com/chap1.html) • Eventually after many iterations (epochs) weights and bias changes to be able to get outputs closer to actual values
15. 15. “2-layer Neural Net”, or “1-hidden-layer Neural Net” “3-layer Neural Net”, or “2-hidden-layer Neural Net” “Fully-connected” layers Sample Neural networks: Architectures
16. 16. Sample Networks – Multi Layered Architecture GoogleNet AlexNet ResNet
17. 17. The Cloud Data Platform for Insights-Driven Enterprises Neural Networks? So What’s New?
18. 18. ILSRVC – Imagenet Deep Learning Disruption
19. 19. Reasons for Success • Vast amounts of Labeled Data – E.g. ImageNet • GPUs – Traditionally used for Gaming – Very Good for Very Large Matrix Operation – 1 Nvidia TitanX 11 Teraflops vs Xeon (500 Giga Flops) – *Google in May 2017, launched TPUs (Tensor Processing Units), custom built for TPUs • Better Activation Functions – RELU, Leaky RELU
20. 20. 22K categories and 14M images www.image-net.org Deng, Dong, Socher, Li, Li, & Fei-Fei, 2009 22 4/4/2017Lecture 1 -Fei-Fei Li & Justin Johnson & Serena Yeung • Animals • Bird • Fish • Mammal • • Invertebrate • • Plants • • Tree • • Flower Food Materials Structures Artifact • Tools • Appliances • Structures • • • Person Scenes • Indoor • Geological Formations SportActivities
21. 21. The Cloud Data Platform for Insights-Driven Enterprises Types of Networks
22. 22. Convolution Neural Network (CNN/Covnets) • Convolution Neural Network – Break Through Architecture started Deep Learning Disruption. • Very Successfully in Images and Video datasets • Used in Text Classification and other use cases
23. 23. Convolution Neural Network Architecture
24. 24. Convolution Neural Network
25. 25. Deep Learning: Learns increasingly complex features pixels edges object parts (combination of edges) object models
26. 26. Recurrent Neural Networks • Need Context • Images do not carry context • Languages – Complex • ”I like spicy food, but it makes me uncomfortable”?
27. 27. Natural Language Processing • Machine Translation – Higher Accuracy – Same Model can do multi-language translation without pre-training on every language • Question Answer System • Word Embeddings • Sentence Completions • Speech to Text (Siri, Android) • Text to Speech • Text Summarization
28. 28. Reinforcement Learning
29. 29. Deep Reinforcement Learning
30. 30. Machine Learning vs Deep Learning • Higher Accuracy Rates – Image Recognition almost as good as humans – Machine Translations as good as Humans in many Western languages – Amazing accurate results with lot of traditionally hard to solve areas • No Hand Crafted Features – Traditional Machine Learning needs handcrafting of Feature Vectors – Deep Learning, No hand crafting of features
31. 31. The Cloud Data Platform for Insights-Driven Enterprises Frameworks
32. 32. Deep Learning Needs • Intensive Mathematical Operations • Working on Multidimensional Matrices • Various Mathematical Operations for Matrix Initialization (Gaussian, etc) • Random Value generation, Initialization functions • Support for processing on Multi-GPUs • Calculating Loss, Annealing, Decay rate • Support for Convolutions, RNN, LSTMs, GRUs • Support Multiple Gradient Descent Algorithms • Ability to persist the models • Need to be able to train fast • Visualize Losses and Accuracy
33. 33. Deep Learning Frameworks Framework Name Adoption Organization Tensorflow High Google Caffe/Caffe2 Medium-High Facebook, UC Berkeley (Good support for Image analysis)- Caffe2 released in 2017 Mxnet Low Amazon. Released in 2017 CNTK Medium (High in Microsoft Users) Microsoft. Good example with Image Identification (COCO dataset) Theano Medium University of Montreal. One of the oldests frameworks.
34. 34. Deep Learning Frameworks Framework Adoption Organization Keras High Google. Extremely popular. Torch/PyTorch Medium - High Open Source. Twitter uses it. Very popular in Non Python user base DeepLearning4J Medium DeepLearning4J. Small company in SF, started in 2014. Good Java and Hadoop support. Loosing grounds to Tensorflow. Chainer Low-Medium Preferred Networks. A japanese company. Applications in IOT and Robotics
35. 35. Deep Learning Frameworks Framework Adoption Organization Neon Low-Medium Intel. Nervana acquired in 2016. Fastest DL Framework BigDL Low Intel. Support for running DeepLearning on Spark. Python Numpy like API. Built in support for Intel MKL libraries. Cloudera Supports CUDA High Nvidia. All frameworks use it and Self Driving Car industry TensorRT Low Nvidia. Optimizes the Deep Learning layers, increasing inference performance.
36. 36. Language Language Adoption Python Very High. Most Common. Works well with numpy, openCV, scikit- learn. Lua (Torch) Medium. Used at Twitter and some universities. C++ Medium. Common with Hardware vendors and Low lever runtime implementations Java Very low. Only among Deeplearning4j users
37. 37. Frameworks • Most Frameworks are similar and do a similar job as listed in Slide 2. • Tensorflow and Keras are the most widely adopted – Large support from Google – Support for Threading and multi GPU – First Class support for Python – Support for HDFS and Google Cloud File System – Distributed compute support
38. 38. Other Env. topics OS Adoption Ubuntu (16 or 14) Very Prevalent as a default OS to be supported Notebooks Jupyter Almost All examples on Jupyter Notebook
39. 39. The Cloud Data Platform for Insights-Driven Enterprises Hardware
40. 40. Hardware Training • 4-8 GPU Nodes (Titanx, K80, P100, Volta*) – On Google this would become TPU in coming months • Multi Cluster (High Speed Network) • Training time (Days to Weeks) Inferencing • Less Compute need • CPUs (No need for GPUs) • Can run on light weight low power devices like “Smart Phones” or “pi devices”
41. 41. The Cloud Data Platform for Insights-Driven Enterprises Deep Learning Ecosystem
42. 42. Major Companies with AI Research Google > 50% Mindshare of the AI Market
43. 43. Competitive Landscape Company Product Remarks Microsoft CNTK https://studio.azureml.net/ - A very comprehensive support for Machine Learning Libraries. - A well designed Interface Azure Cloud is growing very fast. They have actively taken up market share from Amazon IBM Watson Power8 PC with NVLink Historic Dominance with Deep Blue (Chess) and Jeopardy IBM BlueMix IBM uses Watson to Market itself.
44. 44. Azure Machine Learning Studio
45. 45. Competitive Landscape Company Product Remarks Alphabet Google ML Engine Rest API Based Vision API Video Intelligence API Natural language Translation API Deep Mind - Solving Artificial General Intelligence - Impact on Healthcare and Data Center Power Consumption Tensor Processing Unit - Competing with Nvida - Will be offered as a Cloud Service Company with largest Mindshare in Artificial Intelligence. I think Google will be the biggest competitor in the Cloud Business going forward. https://cloud.google.com/products/ Amazon - Apache Mxnet Similar Rest based API as Google Market Leader in Cloud
46. 46. Google Cloud Vision
47. 47. Competitive Landscape Company Product H20.ai Sparkling Water and Deep Water SigOpt Improve ML Models DataRobot Build and Deploy Machine Learning Models Clarifai.ai Image and Video Tagging Crowdflower.ai Dataset preparation for Uber and many companies
48. 48. H20 Clarifai.ai
49. 49. Sample Machine Learning – Life Cycle Get/Prepare Data Build/Edit Experiment Create/Update Model Evaluate Model Results Build ML Model Deploy as Web Service Provision Workspace Get Qubole Subscription Create Cluster Publish an App Integrate with App/Analytics Publish the model Deploy Model as a Web Service Examine the Predictions / Use more production data to fine tune Model
50. 50. The Cloud Data Platform for Insights-Driven Enterprises Challenges
51. 51. Challenges to Deep Learning Success (Andrew Ng) • Data. Among leading AI teams, many can likely replicate others’ software in, at most, 1–2 years. But it is exceedingly difficult to get access to someone else’s data. Thus data, rather than software, is the defensible barrier for many businesses. • Talent. Simply downloading and “applying” open-source software to your data won’t work. AI needs to be customized to your business context and data. This is why there is currently a war for the scarce AI talent that can do this work.
52. 52. Modern Scientist or Stars of Deep Learning
53. 53. Modern Scientist or Stars of Deep Learning
54. 54. Modern Scientists