MXNet Workshop
•
6 gostaram•1,323 visualizações
This document provides an overview of deep learning including: - Deep learning has significantly improved applications in computer vision, speech recognition, and natural language processing over the past 10 years. - It describes some common deep learning models like convolutional neural networks, LSTMs, and feedforward networks. - It also shows how to deploy deep learning models on AWS services like EC2 and provides examples of using MXNet on common deep learning tasks like image classification and recommender systems.
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (13)
Semelhante a MXNet Workshop
Semelhante a MXNet Workshop (20)
Mais de Amazon Web Services
Mais de Amazon Web Services (20)
Último
Último (20)
MXNet Workshop
- 1. What do you need to know about Deep Learning Let’s get our hands dirty
- 2. Not so long ago…
- 3. Today…
- 5. Significantly improve many applications on multiple domains “deep learning” trend in the past 10 years image understanding speech recognition natural language processing … Deep Learning autonomy
- 6. 0.2 -0.1 ... 0.7 Input Output 1 1 1 1 0 1 0 0 0 3 mx.sym.Pooling(data, pool_type="max", kernel=(2,2), stride=(2,2) lstm.lstm_unroll(num_lstm_layer, seq_len, len, num_hidden, num_embed) 4 2 2 0 4=Max 1 3 ... 4 0.2 -0.1 ... 0.7 mx.sym.FullyConnected(data, num_hidden=128) 2 mx.symbol.Embedding(data, input_dim, output_dim = k) Queen 4 2 2 0 2=Avg Input Weights cos(w, queen) = cos(w, king) - cos(w, man) + cos(w, woman) mx.sym.Activation(data, act_type="xxxx") "relu" "tanh" "sigmoid" "softrelu" Neural Art Face Search Image Segmentation Image Caption “People Riding Bikes” Bicycle, People, Road, Sport Image Labels Image Video Speech Text “People Riding Bikes” Machine Translation “Οι άνθρωποι ιππασίας ποδήλατα” Events mx.model.FeedForward model.fit mx.sym.SoftmaxOutput Anatomy of a Deep Learning Model mx.sym.Convolution(data, kernel=(5,5), num_filter=20) Deep Learning Models
- 7. Elastic Compute Cloud (EC2) • C4 Instances • c4.8xlarge On-Demand: • $1.675/hr • GoogleNet inference: • batch size 32 • 262 ims/sec = 3.8 ms/im • 1 million images costs $1.77 • Spot prices are cheaper OS: Linux version 3.13.0-86-generic (buildd@lgw01-51) (gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) ) #131-Ubuntu SMP Thu May 12 23:33:13 UTC 2016. MxNet Tip of tree: commit de41c736422d730e7cfad72dd6afc229ce08cf90, Tue Nov 1 11:43:04 2016 +0800. MKL 2017 Gold update 1 7 6.1 2.4 1.2 0.8 679.5 262.5 79.7 73.9 0 200 400 600 800 AlexNet GoogLeNet v1 ResNet-50 Inception v3 Images/Sec c4.8xlarge MXNet Inference No MKL MKL
- 8. deploy Bigdl on aws Github: github.com/intel-analytics/BigDL http://software.intel.com/bigdl BigDL, A Distributed Deep learning framework for Apache Spark* Deploying BigDL on AWS is super easy! Option 1: Install BigDL on Amazon EMR with Bootstrap action s3://aws-bigdata-blog/artifacts/aws-blog-emr-jupyter/install-jupyter-emr5- latest.sh –bigdl Option 2: Launch Public AMI on EC2 w/Xeon E5 v3 or v4 https://github.com/intel-analytics/BigDL/wiki/Running-on-EC2 Learn how to Classify text, train a Convolutional neural network, visualize the training using Tensorboard using BigDL on AWS. https://aws.amazon.com/blogs/ai/running- bigdl-deep-learning-for-apache-spark-on- aws/
- 9. Launch your notebooks • Follow the instructions on http://bit.ly/MXNet-blog #!/bin/bash -v sudo su - cd ~ export PYTHONPATH=/home/ubuntu/mxnet/python export LD_LIBRARY_PATH=/usr/local/cuda/lib64/ git clone https://github.com/dmlc/mxnet-notebooks jupyter notebook ssh -L 127.0.0.1:8888:127.0.0.1:8888 -i PEM-KEY ec2-user@EC2-DNS
- 10. Any Data Any Problem
- 14. MNIST dataset in t-SNE
- 15. MNIST Notebook
- 16. Gradient Descent
- 17. MNIST Notebook – CNN filter
- 21. MNIST Notebook – Activation Options
- 25. LSTM Notebook
- 26. RNN Examples Image Caption Sentiment Analysis Machine Translation Video LabelingImage Labeling
- 30. MF Example - Embedding
- 31. MF Example - Optimizers
- 33. DSSM Example - Recommender
Notas do Editor
- 1000000 M * $1.675 / hr * 1 hr/3600 s * 1 s / 262.5 imgs = $1.77
- BigDL is a distributed deep learning framework for Apache Spark that was developed by Intel and contributed to the open source community for the purposes of uniting big data processing and deep learning. BigDL helps make deep learning more accessible to the big data community by allowing developers to continue using familiar tools and infrastructure to build deep learning applications. As shown in the diagram on the right, BigDL is implemented as a library on top of Spark, so that users can write their deep learning applications as standard Spark programs. As a result, BigDL can be seamlessly integrated with other libraries on top of Spark—Spark SQL and DataFrames, Spark ML pipelines, Spark Streaming, Structured Streaming, etc.—and can run directly on top of existing Spark or Hadoop clusters. It is super easy to deploy BigDL on AWS. There is a public AMI available to launch BigDL on EC2 Spark instances with Xeon E5 v3 or v4 processors. The Project Git has a detailed tutorial on how to launch this AMI and run the BigDL examples for training various models. The AWS team also partnered with the BigDL project team to develop a tutorial for running BigDL on AWS – we have simplified it with a bootstrap action….you can use the following bootstrap action with the –bigdl argument in an AWS CLI command or in the AWS EMR console…You can follow the tutorial blog we published and try it out. We have seen major advancements in deep learning in the recent years; while the deep learning community continue to push the technology envelope, the appeal of a project like BigDL on Apache Spark is that it helps make these breakthroughs more accessible and convenient to use for data scientists and data engineers (who are not necessarily experts in deep learning technologies).
- NN can take every type of input and map it to every type of classification
- The biological neurological model that is able to learn human intelligence such as vision, language, problem solving and every other human abilities. The brain is based on chemistry models, and it 100 billions of neurons, and it has modes as depression or high adrenaline, which we are still not simulating in NN.
- Deep Structured Semantic Model was developed by MSR, and allows the integration of multiple type of NN output and input to be used together.
- Deep Structured Semantic Model was developed by MSR, and allows the integration of multiple type of NN output and input to be used together.
- Deep Structured Semantic Model was developed by MSR, and allows the integration of multiple type of NN output and input to be used together.