Working with Amazon SageMaker Algorithms for Faster Model Training

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
SageMaker:
Infinitely Scalable Machine Learning
Nick Brandaleone
Solutions Architect at Amazon Web Services

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Customer Running ML on AWS Today

AWS ML Stack
FRAMEWORKS AND INTERFACES
AW S DEEP LEARNING API
Apache MXNet TensorFlowCaffe2 Torch KerasCNTK PyTorch GluonTheano
PLATFORM SERVICES
VISI ON
AWS DeepLensAmazon SageMaker
LANGUA G E
A P P L I C A T I O N S E R V I C E S
Amazon
Rekognition
Amazon
Polly
Amazon
Lex
Amazon
Rekognition
Video
Amazon Transcribe Amazon Translate
Amazon
Comprehend
Alexa for Business
VR/IR Amazon Sumerian
Amazon Kinesis
Video Streams

Deep Learning AMI

Amazon EC2 P3 Instances (October 2017)
• Up to eight NVIDIA Tesla V100 GPUs
• 1 PetaFLOPs of computational
performance – 14x better than P2
• 300 GB/s GPU-to-GPU communication
(NVLink) – 9X better than P2
• 16GB GPU memory with 900 GB/sec peak
GPU memory bandwidth
T h e f a s t e s t , m o s t p o w e r f u l G P U i n s t a n c e s i n t h e c l o u d

A fully managed service that enables data scientists and developers to quickly and easily
build machine-learning based models, and rapidly bring them into production.
Amazon SageMaker

Let’s Review the ML Process

Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
The Machine Learning Process
Re-training

Analysis
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Discovery: The Analysts
Re-training
• Help formulate the right
questions
• Domain Knowledge

Analysis
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Integration: The Data Architecture
Retraining
• Build the data platform:
• Amazon S3
• AWS Glue
• Amazon Athena
• Amazon EMR
• Amazon Redshift
Spectrum

Analysis
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
• Setup and manage
Notebook Environments
• Setup and manage
Training Clusters
• Write Data Connectors
• Scale ML algorithms to
large datasets
• Distribute ML training
algorithm to multiple
machines
• Secure Model artifacts
Why We built Amazon SageMaker: The Model Training Undifferentiated Heavy Lifting

Model Deployment
Monitoring &
Debugging
– Predictions
• Setup and manage Model
Inference Clusters
• Manage and Scale Model
Inference APIs
• Monitor and Debug Model
Predictions
• Models versioning and
performance tracking
• Automate New Model
version promotion to
production (A/B testing)
Why We built Amazon SageMaker: The Model Deployment Undifferentiated Heavy Lifting

Amazon SageMaker
1 2 3 4
I I I I
Notebook Instances Algorithms ML Training Service ML Hosting Service

Highly-optimized
machine learning
algorithms
Amazon SageMaker
BuildPre-built
notebook
instances

Highly-optimized
machine learning
algorithms
One-click training
for ML, DL, and
custom algorithms
BuildPre-built
notebook
instances
Easier training with
hyperparameter
optimization
Train
Amazon SageMaker

One-click training
for ML, DL, and
custom algorithms
Easier training with
hyperparameter
optimization
Highly-optimized
machine learning
algorithms
Deployment
without
engineering effort
Fully-managed
hosting at scale
BuildPre-built
notebook
instances
Deploy
Train
Amazon SageMaker

End-to-End
Machine Learning
Platform
Zero setup Flexible Model
Training
Pay by the second
$
Amazon SageMaker
Build, train, and deploy machine learning models at scale

Amazon ECR
Model Training (on EC2)
Amazon SageMaker
Client application
Training code

Amazon ECR
Trainingdata
Training code Helper code
Client application
Training code
Amazon SageMaker

Amazon ECR
Trainingdata
Modelartifacts
Client application
Inference code
Training code
Amazon SageMaker

Amazon ECR
Model Hosting (on EC2)
Trainingdata
Modelartifacts
Helper codeInference code
Client application
Inference code
Training code
Amazon SageMaker

Amazon ECR
Trainingdata
Modelartifacts
Client application
Inference code
Training code
Inference requestInference response
Inference Endpoint
Amazon SageMaker

Amazon ECR
Trainingdata
Modelartifacts
GroundTruth
Client application
Inference code
Training code
Inference requestInference response
Inference Endpoint
Amazon SageMaker

Key benefits of SageMaker at Intuit
Ad-hoc setup and management
of notebook environments
Limited choices for model
deployment
Competing for compute
resources across teams
Easy data exploration
in SageMaker notebooks
Building around virtualization
for flexibility
Auto-scalable model hosting
environment
From To

Built-in ML Algorithms
Problem Algorithm Learning Type
Discrete Classification,
Regression
Linear Learner Supervised
XGBoost Algorithm Supervised
Discrete Recommendations Factorization Machines Supervised
Image Classification Image Classification Algorithm Supervised
Neural Machine Translation Sequence to Sequence Supervised
Discrete Groupings K-Means Algorithm Unsupervised
Dimensionality Reduction PCA (Principal Component Analysis) Unsupervised
Topic Determination Latent Dirichlet Allocation (LDA) Unsupervised
Neural Topic Model (NTM) Unsupervised,
Neural Network Based
Time Series Forecasting DeepAR Forecasting Supervised

SageMaker instance types
Instance vCPU Mem (GiB)
t2 2-8 4-32
m4 4-64 16-256
m5 2-96 8-384
c4 2-36 4-60
c5 2-72 4-144
P2 (GPU) 4-64 61-732
P3 (GPU) 8-64 61-488

Linear Learner
Regression:
Estimate a real valued function
Binary Classification:
Predict a 0/1 class

K-Means Clustering

Principal Component Analysis (PCA)

Neural Topic Modeling
Encoder: feedforward net
Input term counts vector
Document
Posterior
Sampled Document
Representation
Decoder:
Softmax
Output term counts vector
Perplexity vs. Number of Topic

Time Series Forecasting
Mean absolute
percentage error
P90 Loss
DeepAR R DeepAR R
traffic
Hourly occupancy rate of 963
bay area freeways
0.14 0.27 0.13 0.24
electricity
Electricity use of 370
homes over time
0.07 0.11 0.08 0.09
pageviews
Page view hits
of websites
10k 0.32 0.32 0.44 0.31
180k 0.32 0.34 0.29 NA
One hour on p2.xlarge, $1
Input
Network

Examples
Command Line
SageMaker Notebooks
EMR

Input Data
>> aws --profile <profile>
--region us-west-2
sm create-training-job
--training-job-name kmeans-demo
--algorithm-specification TrainingImage=0123456789.dkr.ecr.us-east-
1.amazonaws.com/kmeanswebscale:latest,TrainingInputMode=File
--role-arn "arn:aws:iam::0123456789:role/demo"
--input-data-config '{"ChannelName": "train", "DataSource":
{"S3DataSource":{"S3DataType": "S3Prefix", "S3Uri":
"s3://kmeans_demo/train", "S3DataDistributionType":
"FullyReplicated"}}, "CompressionType": "None", "RecordWrapperType": "None"}'
--output-data-config S3OutputPath=s3://kmeans_demo/output
--resource-config InstanceCount=2,InstanceType=c4.8xlarge,VolumeSizeInGB=50
--stopping-condition MaxRuntimeInHours=1
From Command Line
Hardware

From SageMaker Notebooks
Parameters
Hardware
Start Training
Host model

From EMR
Start Training
Parameters
Hardware
Apply Model

Using Your Own Algorithm
s a m p l e - n o t e b o o k s / a d v a n c e d _ f u n c t i o n a l i t y / s c i k i t _ b r i n g _ y o u r _ o w n
s a m p l e - n o t e b o o k s / a d v a n c e d _ f u n c t i o n a l i t y / r _ b r i n g _ y o u r _ o w n
Step 1. Package your training or inference code to Docker image
• Make the Docker container nvidia-docker compatible only with CUDA toolkit not NVIDIA drivers
• Dockerfile: describes how to build your Docker container image
• build_and_push.sh: a script that uses the Dockerfile to build your
container images and then pushes it to ECR
• decision_trees: directory which contains the files that will be installed
in the container
• train: training programn
• nginx.conf, predictor.py, serve, wsgi.py : inference

Step 2. Building and Registering the container
• build_and_push.sh script does this
• ‘docker build’ to build a Docker image
• ‘docker push’ to push the container image into Amazon Elastic Container Registry (
ECR)
Step 3. Training the algorithm in Amazon SageMaker

Step 4. Deploy the model to SageMaker hosting
• deploy call does this
• an instance count, instance type, and optionally serializer and deserializer functions

Amazon ECR
30 50
10 10
ProductionVariant
Model Artifacts
Inference Image
Model versions
Versions of the same
inference code saved in
inference containers.
Prod is the primary
one, 50% of the traffic
must be served there! Create an Endpoint from
one EndpointConfiguration
EndpointConfiguration
Inference Endpoint
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker
InstanceType: c3.4xlarge
InitialInstanceCount: 3
ModelName: prod
VariantName: primary
InitialVariantWeight: 50

Amazon SageMaker
!GO BUILD!
End-to-End Managed ML Platform

aws.amazon.com/activate
Everything and Anything Startups
Need to Get Started on AWS

Working with Amazon SageMaker Algorithms for Faster Model Training

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Working with Amazon SageMaker Algorithms for Faster Model Training

Semelhante a Working with Amazon SageMaker Algorithms for Faster Model Training (20)

Mais de Amazon Web Services

Mais de Amazon Web Services (20)

Working with Amazon SageMaker Algorithms for Faster Model Training