Machine learning (ML) provides innovation for every business. Until recently, developing ML models took time and effort, making it difficult for developers to get started. In this session, we demonstrate how Amazon SageMaker—a fully managed service that enables developers to build, train, and deploy ML models at scale—overcomes these barriers. We review its capabilities across data labeling, model building, model training, tuning, and production hosting. We also discuss the details of the modules within Amazon SageMaker, assisting developers through the steps of the ML workflow.

1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker: ML for every developer
and data scientist
Urvashi Chowdhary
Senior product manager
Amazon SageMaker, AWS
A I M 2 0 2

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Customer-focused
90%+ of our ML roadmap is
defined by customers
Pace of innovation
200+ new ML launches
and major feature updates last year
Breadth and depth
A wide range of AI and ML services
Multi-framework
Support for the most
popular frameworks
Security & analytics
Deep set of security
with robust encryption
and analytics
Embedded R&D
Customer-centric approach
Our approach to machine learning

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
M L F R A M E W O R K S &
I N F R A S T R U C T U R E
The Amazon ML stack: Broadest & deepest set of capabilities
A I S E R V I C E S
Vision | Documents | Speech | Language | Chatbots | Forecasting | Recommendations
M L S E R V I C E S Data labeling | Pre-built algorithms & notebooks | One-click training and deployment
Build, train, and deploy machine learning models fast
Easily add intelligence to applications without machine learning
skills
Flexibility & choice, highest-performing infrastructure
Support for ML frameworks | Compute options purpose-built for ML

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
M L F R A M E W O R K S &
I N F R A S T R U C T U R E
The Amazon ML stack: Broadest & deepest set of capabilities
A I S E R V I C E S
A M A Z O N
R E K O G N I T I O N
I M A G E
A M A Z O N
P O L L Y
T R A N S C R I B E T R A N S L A T E C O M P R E H E N D &
C O M P R E H E N D
M E D I C A L
A M A Z O N
L E X
A M A Z O N
R E K O G N I T I O N
V I D E O
Vision Speech Chatbots
A M A Z O N S A G E M A K E R
B U I L D T R A I N
A M A Z O N
F O R E C A S T
A M A Z O N
T E X T R A C T
A M A Z O N
P E R S O N A L I Z E
D E P L O Y
Pre-built algorithms & notebooks
Data labeling (G R O U N D T R U T H )
One-click model training & tuning
Optimization ( N E O )
One-click deployment & hosting
M L S E R V I C E S
F r a m e w o r k s I n t e r f a c e s I n f r a s t r u c t u r e
E C 2 P 3
& P 3 d n
E C 2 C 5 F P G A s A W S I o T
G R E E N G R A S S
A M A Z O N
E L A S T I C
I N F E R E N C E
Models without training data (REINFORCEMENT LEARNING)
Algorithms & models ( A W S M A R K E T P L A C E )
Language Forecasting Recommendations
RL Coach

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker
Build, train, and deploy machine learning models quickly & easily, at scale

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker – Launched at re:Invent 2017
1
2
3

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
1
2
3
Amazon SageMaker:
Build, train, and deploy ML models at scale

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker:
Build, train, and deploy ML models at scale

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker:
Build, train, and deploy ML models at scale

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker:
Build, train, and deploy ML models at scale

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker:
Build, train, and deploy ML models at scale

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker:
Build, train, and deploy ML models at scale

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Ongoing enhancements to Amazon SageMaker
MXNet 1.3 container | CloudTrail integration for audit logs | TensorFlow 1.7 Containers | Automatic Model Tuning—Add/Delete tags | Jupyter Notebooks IP Filtering
Region expansion to SFO | Image Classification Multi-label Support | TensorFlow and MXNet Containers—Open Sourcing and Local Mode | PyTorch pre-built container
Region expansion to PDT | Batch customer VPC | PCI DSS Compliance | XGBoost Instance Weights | NTM—vocab, metrics, and subsampling
Anomaly Detection (Random Cut Forest) Algorithm | Deep AR algorithm | SageMaker region expansion to ICN | Hyperparameter tuning job cloning on the console
Automatic scaling console | PyTorch 1.0 container | Customer VPC support for training and hosting | AWS PrivateLink support for SageMaker inferencing APIs
Horovod support in TensorFlow Container | Variable sizes for notebook EBS volumes |nbexample support in SageMaker notebook instances | Tag-based access control
Automatic Model Tuning—early stopping | IP Insights algorithm | Chainer 4.1 Container | Region expansion to SIN Built-in Algorithms Pipe Mode Support
TensorFlow 1.8 Container | Region expansion to FRA | Training job cloning in console | Algorithm Pipe mode enhancements | Pipe mode support for text, recordIO, and images
TensorFlow 1.5, MXNet 1.0, and CUDA 9 Support | DeepAR Algorithm Enhancements | Linear Learner Multi-class Classification | TensorFlow 1.10 Container
Region expansion to YUL | BlazingText Algorithm | Batch KMS | k-nearest neighbors | Object detection |Chainer pre-built container | Apache Airflow integration
Region expansion to BOM | GDPR compliance | BlazingText Enhancements | TensorFlow 1.9 Container | Notebook bootstrap script
Amazon SageMaker Hosting custom header attribute | Metrics Support in Training Jobs | Object2vec | TensorFlow container enhancements | CloudFormation support
AWS PrivateLink support for SageMaker Control Plane | MXNet 1.2 Container | HIPAA compliance | Ground Truth | Python SDK Marketplace support
Git integration for SageMaker notebooks | Pipe mode support for TensorFlow | ml.p3.2xlarge notebook instances | Internet-free notebook instances
Semantic segmentation algorithm | SageMaker Reinforcement Learning support | Linear Learner Improvements | SageMaker Batch Transform
Region expansion to NRT | High Performance I/O streaming in PIPE Mode | Pause/resume for active learning algorithms | Pre-built scikit-learn container
Step Functions for SageMaker | KMS support for training and hosting | Incremental learning algorithm enhancements | TensorFlow 1.11 container | NTM feature release
Deep Learning Compiler | ONNX Support for Frameworks and Algorithms |Full instance type support | Pipe mode CSV support | Region expansion to LHR
Incremental training platform support | Login anomaly detection algorithm | Serial inference pipeline | Experiment Management | Region expansion to SYD
MXNet container enhancements | Automatic Model Tuning | Automatic Model Tuning—incremental tuning | Spark MLeap 1P container
TensorFlow 1.6 and MXNet 1.1 Containers | Region expansion to SIN | Mead Notebook AWS PrivateLink Support | Linear Learnersupport

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Collect and prepare
training data
Choose and optimize
your
ML algorithm
Train and
tune models
Set up and
manage
environments
for training
Deploy model
in production
Scale and manage
the production
environment
1
2
3
1
2
3
Amazon SageMaker:
Build, train, and deploy ML models at scale

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Successful models require high-quality data

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Successful models require high-quality data

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker Ground Truth
Build highly accurate training datasets and reduce data labeling costs
by up to 70% using machine learning

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker Ground Truth
How it works

19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Raw data Human
annotations
Amazon SageMaker Ground Truth
How it works

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker Ground Truth
How it works

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker Ground Truth
How it works

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker Ground Truth
How it works

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
K e y f e a t u r e s
Automatic labeling via
machine learning
Ready-made and
custom workflows
Label
management
Private and public
human workforce
Amazon SageMaker Ground Truth
Label machine learning training data easily and accurately

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Collect and prepare
training data
Choose and optimize
your
ML algorithm
Train and
tune model
Set up and
manage
environments
for training
Deploy model
in production
Scale and manage
the production
environment
Amazon SageMaker
Ground Truth
Amazon SageMaker:
Build, train, and deploy ML models at scale

25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AWS Marketplace for Machine Learning
Over 200 algorithms and models that
can be deployed directly to Amazon SageMaker

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AWS Marketplace for Machine Learning
ML algorithms and models availableinstantly
Subscribe in a
single click
Available in
Amazon SageMaker
K e y f e a t u r e s
Automatic labeling via machine learning
IP protection
Automated billing and metering
Browse or search
AWS Marketplace
S e l l e r s
Broad selection of paid, free, and
open-source algorithms and models
Data protection
Discoverable on your AWS bill
B u y e r s

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Over 200 algorithms and models
Natural language
processing
Grammar & parsing Text OCR Computer vision
Named entity
recognition
Video classification
Speech recognition Text-to-speech Speaker identification Text classification 3D images Anomaly detection
Text generation Object detection Regression Text clustering Handwriting recognition Ranking
A v a i l a b l e a l g o r i t h m s & m o d e l s
S e l e c t e d v e n d o r s

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Collect and prepare
training data
Choose and optimize
your
ML algorithm
Train and
tune models
Set up and
manage
environments
for training
Deploy model
in production
Scale and manage
the production
environment
Amazon EC2 P3dn
Instances
Amazon SageMaker
Ground Truth Amazon Elastic Inference
AWS Marketplace for
Machine Learning
Amazon SageMaker:
Build, train, and deploy ML models at scale

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Model optimization is extremely complex

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker Neo
Train once, run anywhere with 2x the performance

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker Neo: Train once, run anywhere
Neo

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker Neo
Train once, run anywhere with 2x the performance
K E Y F E A T U R E S
Open-source device runtime and compiler,
1/10 the size of original frameworks

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Machine Learning is a highly collaborative
process every step of the way

34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker workflows
Experiment management
Organize, track, and evaluate
model training experiments with
Amazon SageMaker search

35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Experiment management Automation
Organize, track, and evaluate
model training experiments with
Amazon SageMaker search
Use AWS Step Functions to
automate end-to-end workflows
Integrate with Apache Airflow
Amazon SageMaker workflows

36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker workflows
Experiment Management Automation Collaboration
Organize, track, and evaluate
model training experiments with
Amazon SageMaker search
Use AWS Step Functions to
automate end-to-end workflows
Integrate with Apache Airflow
Link GitHub, AWS CodeCommit,
and self-hosted Git repositories to
notebooks
Clone public and private
repositories
Secure information with IAM,
LDAP, and AWS Secrets Manager

37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Sowhat’s nextfor
machine learning?
How do youteach machine learning models to make decisions
when thereis no training data?

38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Introducing reinforcement learning
Reinforcement learning
(RL)
Supervised learning
(ASR, computer vision)
Unsupervised learning
(anomaly detection,
identifying text topics)
Amount of labeled training data required
Complexityofdecisions

39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
What is an RL environment?
Representation
of the real world
Programmed
to represent real-
world conditions
Enables interaction
with user or a
computer program
Dynamic and updates
itself based on the
interactions and
programmed behavior

40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
RL models learnhow to makedecisions
to accomplish tasks

41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
This makes RL applicable in many domains,
not just gaming
Robotics Industrial
control
HVAC Autonomous
vehicles
NLP Operations Finance Resource
allocation
Advertising
Online content
delivery
Achieve outcomes, not decisions

42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How does RL work?

43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How does RL work?
Simulation
environment
Scoring function RL algorithm
Extremely complex Expensive
Effectively out of reach

44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker RL
New machine learning capabilities in Amazon SageMaker to
build, train, and deploy with reinforcement learning

45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker RL
Reinforcementlearningfor every developer and data scientist
2D & 3D physics
environments and
OpenGym support
Support Amazon Sumerian, AWS
RoboMaker, and the open source
Robotics Operating System (ROS)
project
Fully
managed
Example notebooks
and tutorials
K e y f e a t u r e s

46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Predictions drive
complexity and cost in
production
Inference
(prediction)
90%
Training
10%

47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
The challenges of inference in production
One size does not fit allLow utilization and high costs
How do we optimize resources and reduce costs?

48. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon Elastic Inference
Reduce deep learning inference costs by up to 75%

49. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon Elastic Inference
Lower inference costs Match capacity to
demand
Available between
1 to 32 TFLOPS per
accelerator
Key features
Integrated with
Amazon EC2 and
Amazon SageMaker
Support for TensorFlow, Apache
MXNet, and ONNX
with PyTorch coming soon
Single and
mixed-precision
operations
Reduce deep learning inference costs up to 75%

50. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How does Amazon Elastic Inference work with
Amazon SageMaker?
Amazon SageMaker notebooks
❖ Prototype deployments with
Notebooks in local mode
Amazon SageMaker hosted endpoints
❖ Scale endpoints with low-cost
Elastic Inference acceleration

51. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Model support
Amazon EI enabled TensorFlow Serving
and Apache MXNet
ONNX
Amazon EI enabled
TensorFlow Serving
Amazon EI
enabled Apache
MXNet
Applied using
Apache MXNet
❖ Auto discovery of accelerators
❖ IAM-based authentication
❖ Available via: the AWS Deep
Learning AMIs, for download via
Amazon S3 and automatically through
Amazon SageMaker

52. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Recap: Amazon SageMaker
Collect and prepare
training data
Train and
tune models
Set up and
manage
environments
for training
Deploy models
in production
Scale and manage
the production
environment
Amazon EC2 P3
Instances
Amazon SageMaker RL
Amazon SageMaker
Ground Truth
Amazon Elastic
Inference
AWS Marketplace for
Machine Learning
Amazon SageMaker
Neo

53. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
The Amazon ML stack: Broadest & deepest set of capabilities
AI SERVICES
Vision
A m a z o n
R e k o g n i t i o n
I m a g e
A m a z o n
R e k o g n i t i o n
V i d e o
A m a z o n
T e x t r a c t
Speech
A m a z o n
P o l l y
T r a n s c r i b e
Language Chatbots Forecasting Recommendations
T r a n s l a t e C o m p r e h e n d
&
C o m p r e h e n d
M e d i c a l
A m a z o n
L e x
F o r e c a s t A m a z o n
P e r s o n a l i z e
ML SERVICES
A M A Z O N
S A G E M A K E R
G R O U N D T R U T H
N O T E B O O K S
A L G O R I T H M S
A W S M A R K E T P L A C E
R E I N F O R C E M E N T
L E A R N I N G
T R A I N I N G
O P T I M I Z A T I O N
( N E O )
D E P L O Y M E N T
H O S T I N G
ML FRAMEWORKS &
INFRASTRUCTURE
Frameworks Interfaces Infrastructure
E C 2 P 3
& P 3 d n
E C 2 C 5 F P G A s
A W S I o T
G r e e n g r a s s
E L A S T I C
I N F E R E N C E
RL Coach

54. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Additional resources
Amazon SageMaker product page
https://aws.amazon.com/sagemaker
Amazon SageMaker on the AWS Management Console
https://console.aws.amazon.com/sagemaker
Amazon SageMaker blogs
https://aws.amazon.com/blogs/machine-learning/category/artificial-intelligence/sagemaker
Amazon SageMaker 10-minute tutorial
https://aws.amazon.com/getting-started/tutorials/build-train-deploy-machine-learning-model-sagemaker

55. Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Urvashi Chowdhary
urvashic@amazon.com