深入淺出 AWS AI

1© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved |
Young Yang, AI/ML Specialist Solutions Architect
beyoung@amazon.com
AWS Machine Learning
Centerpiece for digital
transformation

Put machine learning in the hands of all developers
機器學習像是 20 - 30 年前的資料庫，是一門顯學。
如今每個開發者可以輕易地上手 SQL Statement，有許多
的工具例如SQL Profiler 幫助開發者調教性能。

Easier to build Easier to scale Easier to apply

The AWS ML Stack
Broadest and most complete set of Machine Learning capabilities
VISION SPEECH TEXT SEARCH CHATBOTS PERSONALIZATION FORECASTING FRAUD DEVELOPMENT CONTACT CENTERS
Ground
Truth
Augmented
AI
ML
Marketplace
Neo
Built-in
algorithms
Notebooks Experiments
Model
training &
tuning
Debugger Autopilot
Model
hosting
Model Monitor
Deep Learning
AMIs & Containers
GPUs &
CPUs
Elastic
Inference
Inferentia
(Inf1 instance)
FPGA
Amazon
Rekognition
Amazon
Polly
Amazon
Transcribe
+Medical
Amazon
Comprehend
+Medical
Amazon
Translate
Amazon
Lex
Amazon
Personalize
Amazon
Forecast
Amazon
Fraud Detector
Amazon
CodeGuru
AI SERVICES
ML SERVICES
ML FRAMEWORKS & INFRASTRUCTURE
Amazon
Textract
Amazon
Kendra
Contact Lens
For Amazon Connect
SageMaker Studio IDE
NEW
NEW! NEW! NEW! NEW!
NEW!
NEW! NEW! NEW! NEW! NEW!
Amazon SageMaker

Machine Learning is a
Journey

Finding the right business use
cases that could benefit from ML

Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Retraining

老闆的期待
Image credit

請問大家對於客服/
聊天機器人的使用經驗?

老闆對於 AI / ML / Big Data 的想像
source

如何選擇ft? 目標：求使得Obj最小的ft
但實務上..

Output
Know or
Estimated values Models
Input parameters
• Yes/No
• Multi-classifications
• Clustering
• Value
• Vectors
What’s your input?
What’s your output to help the measure of
correctness and efficiency
How to frame business problems into machine learning?

Common Problems in Machine Learning
• Finding the right business use cases that could benefit from ML
• Skills gap—not enough people can build ML models
• ML model building is a time-consuming and complex process

AWSMachineLearningEmbarkProgram
ENGAGEMENT
Get hands-on
with DeepRacer
TRAINING
In-person training
DISCOVERY
Cross-functional
workshops
DEVELOPMENT
ML Solutions Lab

Defect Detection w/ML
Problem
FST seek yield improvement in their silicon wafer factories.
The current defect detection process is good but it involve
significant human inspection efforts.
Solution
FST would like to partner with AWS to push the yield
envelope with AI/ML. We conducted ML workshop and
hackathon to educate the teams on the latest AWS
technologies. We then worked together to create the ML
silicon defect detection model using tens of thousands of
wafer images for training.
Impact
We finally create a ML defect detection model with 99%+
detection rate (aka., Recall Rate), improved yield and
reduced the human inspection efforts by half.
“AWS not only is the ML expert with advanced capable tools
but also our partner to show us how to use them to improve
our production operations.
AWS 不只是提供先進機器學習工具的專家，更是教導我們如
何在應用的合作伙伴。
Jason Lin
Chairman, Formosa Sumco Technology Corporation

Machine
Learning
D-Link is a multi-national networking equipment
manufacturing company specializing in designing
and developing products such as Wi-Fi routers, IP
cameras, and smart home devices for consumers,
businesses, and service providers. D-Link provides
networking solutions integrating capabilities in
switching, wireless, broadband, IP surveillance,
and cloud-based network management.
• D-Link was looking for an
opportunity to offer a new,
innovative service to their
clients as part of their
existing video surveillance
system.
• D-Link customers can apply
this ML model in multiple
surveillance scenarios. For
example, users may open
their live video streams and
see a suspicious person
loitering around. The user
can tap on the person in the
video and select “search
object” to see if the same
person has been loitering
around in other areas during
other parts of the day. If
necessary, the user can then
contact the police.
“We specialize in delivering the smartest, most innovative products & services to our
customers. Machine learning provides us with new perspectives and methodologies in order
to achieve this.”
– - Jason Syu, Director of Cloud Solutions Development
D-Link leverages video recognition ML to innovate surveillance
D-Link International Pte Ltd.
Computers & Electronics
Taiwan ROC
2,000+
www.dlink.com
• Using Amazon SageMaker,
the AWS team developed
an image recognition
model that searches video
feeds to locate relevant
objects and re-identify
people.
• Both the image containing
the relevant object or
person and the videos to
be scanned served as
inputs to the customized
recognition model. The
model then verifies
whether the same object or
person in the submitted
image appears in any of
the video feeds.

Data is King:
80% of Your Time Is On Data

Analysis
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Retraining

NEW: CUSTOM LABELS FOR AMAZON REKOGNITION
Machine: 96.9%
Wheel: 95.5%
Bracket: 80.3%
Prop shaft
mid bearing
Clutch
pressure
plate
Plant: 99.2%
Corn: 95.3%
Food: 95.3%
Vegetable: 95.3%
Sweet Corn Field Corn
General identification Specialized identification

以影像辨識判斷貓狗，需要千到萬張的圖片做訓練。
入門初級進階專業
100 ~1,000
張影像
辨識有沒有
1,000 ~10,000
張影像
貓或狗?
10,000 ~ 1M
(retriever vs labrador)
拉布拉多黃金獵犬? This is My Dog
1M＋
張影像
(or IoT)
如需進階的檢驗，例如拉布拉多 vs 黃金獵犬，則需更大量的資料訓練
Image source

Amazon SageMaker Processing
Fully Managed Data Processing and Model Evaluation
SageMaker Processing
Containers
Amazon Simple Storage
Service
/opt/ml/processing/input
/opt/ml/processing/output
s3://path/to/my/input-data preprocessing.py
sourcegithub
script_processor.run(code='preprocessing.py',
inputs=[ProcessingInput(
source='s3://path/to/my/input-data.csv',
destination='/opt/ml/processing/input')],
outputs=[ProcessingOutput(source='/opt/ml/processing/output/train'),
ProcessingOutput(source='/opt/ml/processing/output/validation'),
ProcessingOutput(source='/opt/ml/processing/output/test')]

Training:
No General Purpose Intelligence
No One Fit for All

Analysis
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Retraining

Model development
Feature engineering
Model operations
Versioning
Architecture
Job scheduler
Compute resources
Data warehouse
How much
data scientist
cares
How much
infrastructure
is needed
Data Scientist Productivity

The AWS ML Stack
Broadest and most complete set of Machine Learning capabilities
VISION SPEECH TEXT SEARCH CHATBOTS PERSONALIZATION FORECASTING FRAUD DEVELOPMENT CONTACT CENTERS
Ground
Truth
Augmented
AI
ML
Marketplace
Neo
Built-in
algorithms
Notebooks Experiments
Model
training &
tuning
Debugger Autopilot
Model
hosting
Model Monitor
Deep Learning
AMIs & Containers
GPUs &
CPUs
Elastic
Inference
Inferentia
(Inf1 instance)
FPGA
Amazon
Rekognition
Amazon
Polly
Amazon
Transcribe
+Medical
Amazon
Comprehend
+Medical
Amazon
Translate
Amazon
Lex
Amazon
Personalize
Amazon
Forecast
Amazon
Fraud Detector
Amazon
CodeGuru
AI SERVICES
ML SERVICES
ML FRAMEWORKS & INFRASTRUCTURE
Amazon
Textract
Amazon
Kendra
Contact Lens
For Amazon Connect
SageMaker Studio IDE
NEW
NEW! NEW! NEW! NEW!
NEW!
NEW! NEW! NEW! NEW! NEW!
Amazon SageMaker

Ground
Truth
Algorithms
& Frameworks
Collaborative
Notebooks
ExperimentsDistributed
Training
Deployment, Hosting,
& Monitoring
Amazon SageMaker Autopilot
Build, Train, Deploy Machine Learning Models Quickly at Scale
Reinforcement
Learning
Tuning
& Optimization
SageMaker Studio
Marketplace
for ML
Amazon SageMaker
Debugger

Amazon SageMaker
built-in algorithms
Supported
frameworks
AWS Marketplace
algorithms
Model
Data Data Data Data
Orchestration
Built-in algorithms
AmazonSageMaker
Model Model Model
Orchestration
AmazonSageMaker
Custom
script Algorithms or
models
Custom script on
supported framework
BYO algorithm
and framework
18 built-in high-
performance algorithms
Supported frameworks:
Apache MXNet, TensorFlow,
Scikit-learn, PyTorch, Chainer
Docker containers with your
own algorithms and frameworks
Third-party algorithms
and models
Supported
frameworks
Orchestration
AmazonSageMaker
Custom script
and custom
framework
Orchestration
AmazonSageMaker
Amazon SageMaker training

Amazon SageMaker
Addressing challenges to machine learning
Amazon SageMaker
Studio
Amazon SageMaker
Notebooks
(Preview) Amazon SageMaker
Debugger
Amazon SageMaker
Experiments
Amazon SageMaker
Model Monitor
Amazon SageMaker
Autopilot

Amazon SageMaker
Amazon SageMaker
Studio
Amazon SageMaker
Notebooks
Debugger
Amazon SageMaker
Experiments
Amazon SageMaker
Model Monitor
Amazon SageMaker
Autopilot

Before Studio
1. In data science teams, multiple scientists often share a single AWS account.
Data scientists don’t want to use the AWS Console and their IT team doesn’t
want them to.
2. IT needs to grant data scientist console access rights for different projects:
SageMaker, IAM execution role, s3 buckets, and etc.
3. Notebook instances are EC2 if you screw up the package installation.
4. Hard to collaborate with other scientists.

Login via URL not AWS console
environment
Know who can access this studio.
Easy to manage the team member
instead AWS IAM user.

In the new Amazon SageMaker Notebooks, all users in an
account get a home directory independent of the compute.

Amazon SageMaker
Amazon SageMaker
Studio
Amazon SageMaker
Notebooks
Debugger
Amazon SageMaker
Experiments
Amazon SageMaker
Model Monitor
Amazon SageMaker
Autopilot

Before
Amazon CloudWatch AWS Lambda Amazon Elasticsearch
Service

Amazon SageMaker
Amazon SageMaker
Studio
Amazon SageMaker
Notebooks
Debugger
Amazon SageMaker
Experiments
Amazon SageMaker
Model Monitor
Amazon SageMaker
Autopilot

How does Amazon SageMaker Debugger Work
Training in
progress
Analysis in
progress
Customer’s S3 Bucket
Amazon
CloudWatch Event
Amazon SageMaker
Amazon SageMaker
Studio Visualization
Amazon SageMaker
Notebook
Action  Stop the training
Action  Analyze using
Debugger SDK
Action  Visualize Tensors
using charts
• No code change is necessary to emit debug data with built in algorithms and custom training script
• Analysis occurs real time as data is emitted making real time alerts possible

Amazon SageMaker
Amazon SageMaker
Studio
Amazon SageMaker
Notebooks
Debugger
Amazon SageMaker
Experiments
Amazon SageMaker
Model Monitor
Amazon SageMaker
Autopilot

Customers faced a false choice
DIY model training
• Manual effort by experts
• Fully controlled and
auditable
• Experts make tradeoff
decisions
• Gets better over time with
experience
Automated ML
• Accessible to experts and
non-experts alike
• No visibility into the training
process
• Can’t make tradeoffs
between accuracy and other
characteristics

Customers now have a better choice
Amazon SageMaker
AutopilotDIY model training
• Manual effort by experts
• Fully controlled and
auditable
• Experts make tradeoff
decisions
• Gets better over time with
experience
Automated ML
• Accessible to experts and
non-experts alike
• No visibility into the training
process
• Can’t make tradeoffs
between accuracy and other
characteristics

How it works

Under the hood
1 7 250

Generate the Codes and Notebooks for you
Data Exploration
Candidate Definition Notebook

#
Model Accuracy Latency Model Size
1
churn-xgboost-1756-013-33398f0 95% 450 ms 9.1 MB
2 churn-xgboost-1756-014-53facc2 93% 200 ms 4.8 MB
3
churn-xgboost-1756-015-58bc692 92% 200 ms 4.3 MB
4 churn-linear-1756-016-db54598 91% 50 ms 1.3 MB
5 churn-xgboost-1756-017-af8d756 91% 190 ms 4.2 MB
Model training involves tradeoffs Batch
Transform
Web
Realtime
IoT
Devices
No General Purpose Intelligence
No One Fit for All

Feedback Loop and Automation

Analysis
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Retraining

Feed Back Loop is Important

Easily implement human review of
machine learning predictions
Introducing Amazon Augmented AI
+
ML and humans working together

How Amazon A2I works
Client application
sends input data
AWS AI Service or
custom ML model
makes predictions
Results stored
to Amazon S3
1 2
6
4 Low-confidence
predictions sent for
human review
3
High-confidence predictions
returned immediately to client
application
5 Reviews consolidated
using Amazon A2I answer
consolidation algorithms
Client
application

Automation: Deploy as fast as you can
Deploy to
a device
Train with
SageMaker algorithms
Train with your
own algorithms
Train with
TensorFlow
or MXNet
or PyTorch
A/B TestOptimize
your models
Deploy to
the Cloud
BYO
Rinse and repeat for every device, every model change

The More Automation, The More Innovation
Infrastructure
Support
Innovation
Infrastructure
Support
Innovation
Innovation
Support
✅
automate automate

Take Away

Analysis
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Retraining
Output
Know or
Estimated values Models
Input parameters
• Yes/No
• Multi-classifications
• Clustering
• Value
• Vectors
What’s your input?
What’s your output to help the measure of
correctness and efficiency
How to frame business problems into machine learning?
AWSMachineLearningEmbarkProgram
ENGAGEMENT
Get hands-on
with DeepRacer
TRAINING
In-person training
DISCOVERY
Cross-functional
workshops
DEVELOPMENT
ML Solutions Lab
New
NEW: CUSTOM LABELS FOR AMAZON REKOGNITION
Machine: 96.9%
Wheel: 95.5%
Bracket: 80.3%
Prop shaft
mid bearing
Clutch
pressure
plate
Plant: 99.2%
Corn: 95.3%
Food: 95.3%
Vegetable: 95.3%
Sweet Corn Field Corn
General identification Specialized identification
New

Amazon SageMaker Processing
Fully Managed Data Processing and Model Evaluation
SageMaker Processing
Containers
Amazon Simple Storage
Service
/opt/ml/processing/input
/opt/ml/processing/output
s3://path/to/my/input-data pr epr ocessi ng. py
sourcegithub
script_processor.run(code='preprocessing.py',
inputs=[ProcessingInput(
source='s3://path/to/my/input-data.csv',
destination='/opt/ml/processing/input')],
outputs=[ProcessingOutput(source='/opt/ml/processing/output/train'),
ProcessingOutput(source='/opt/ml/processing/output/validation'),
ProcessingOutput(source='/opt/ml/processing/output/test')]
New
Ground
Truth
Algorithms
& Frameworks
Collaborative
Notebooks
ExperimentsDistributed
Training
Deployment, Hosting,
& Monitoring
Build, Train, Deploy Machine Learning Models Quickly at Scale
Reinforcement
Learning
Tuning
& Optimization
SageMaker Studio
Marketplace
for ML
Amazon SageMaker
Debugger
Amazon SageMaker
Amazon SageMaker
Studio
Amazon SageMaker
Notebooks
Debugger
Amazon SageMaker
Experiments
Amazon SageMaker
Model Monitor
Amazon SageMaker
Autopilot
New
Easily implement human review of
machine learning predictions
Introducing Amazon Augmented AI
+
ML and humans working together
New

Data SelectionData Selection
Video
Data Prep
Data Cleansing
(Missing Values,
Normalization, etc..)
Databases
(Join, Filter, etc..)
Dim Reduce
(ID Salient Features)
Annotation
(Label the ground
truth)
Model Selection
Regression
Classification
Clustering
Model Training
Connectionists
(Neural Networks)
Symbolists
(Logic)
Analogists
(SVMs)
Bayesians
(Graphical
Networks)
Evolutionists
(Genetic Agents)
Boosters
(Boosted Trees)
Model Evaluation
HPO
(Non-learned
parameter tuning)
Cross Validation
(Generalization?)
Metrics
(Bias, Variance, F1
Score)
Regularization
(Dropout, L2, etc..)
Model Deployment
Portability
Ensemble
New Predictions
ML is still too complicated for everyday developers

Machine learning is a journey
Tip 1: Find the right business use cases that could benefit
from ML
Tip 2: Data is King - 80% of Your Time Is On Data
Tip 3: Training - No General Purpose Intelligence,
No One Fit for All
Tip 4: Feedback Loop and Automation

customer
value

HOW WE CAN HELP
• Brainstorming
• Custom modeling
• Training
• Work side-by-side with Amazon experts
ML Solutions Lab
• Practical education on ML for new
and experienced practitioners
• Based on the same material used
to train Amazon developers
Machine Learning
Training and Certification

數據 + 運算 + 持續修正 =人工智慧
Data + Model + Continuously Feedback Loop = AI

Thank You
Young Yang, AI/ML Specialist Solutions Architect
beyoung@amazon.com

深入淺出 AWS AI

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to 深入淺出 AWS AI

Similar to 深入淺出 AWS AI (20)

More from Amazon Web Services

More from Amazon Web Services (20)

深入淺出 AWS AI