Mais conteúdo relacionado Semelhante a SageMaker Algorithms Infinitely Scalable Machine Learning (20) Mais de Amazon Web Services (20) SageMaker Algorithms Infinitely Scalable Machine Learning1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Pop-up Loft
Amazon SageMaker Algorithms:
Infinitely Scalable Machine Learning
Nick Brandalone
Solutions Architect , AWS Solutions Architecture
2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
What is Amazon SageMaker?
Exploration Training
Hosting
3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
The Amazon Machine Learning Stack
FRAMEWORKS & INTERFACES
Caffe2 CNTK
Apache
MXNet
PyTorch TensorFlow Chainer Keras Gluon
AWS Deep Learning AMIs
Amazon SageMaker
Rekognition Transcribe Translate Polly Comprehend Lex
AWS
DeepLens
EDUCATION
PLATFORM SERVICES
APPLICATION SERVICES
Amazon Mechanical Turk
4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Challenges in Machine Learning
5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Machine Learning
6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Large Scale Machine Learning
7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Large Scale Machine Learning
8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Our Customers use ML at a massive scale
“We collect 160M events
daily in the ML pipeline and
run training over the last
15 days and need it to
complete in one hour.
Effectively there's 100M
features in the model”
Valentino Volonghi, CTO
“We process 3 million ad
requests a second, 100,000
features per request. That’s
250 trillion per day. Not
your run of the mill Data
science problem!”
Bill Simmons, CTO
“Our data warehouse is
100TB and we are
processing 2TB daily. We're
running mostly gradient
boosting (trees), LDA and
K-Means clustering and
collaborative filtering.“
Shahar Cizer Kobrinsky, VP
Architecture
9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cost vs. Time
$$$$
$$$
$$
$
Minutes Hours Days Weeks Months
10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cost vs. Time
$$$$
$$$
$$
$
Minutes Hours Days Weeks Months
Single
Machine
11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cost vs. Time
$$$$
$$$
$$
$
Minutes Hours Days Weeks Months
Single
Machine
Distributed, with
Strong Machines
12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cost vs. Time
$$$$
$$$
$$
$
Minutes Hours Days Weeks Months
Single
Machine
Distributed, with
Strong Machines
13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Model Selection
1
1
14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Incremental Training
2
3
1
2
15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Production Readiness
Data/Model Size
Investment
16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Production Readiness
Data/Model Size
Investment Reasonable
Investment Level
17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Production Readiness
Data/Model Size
Investment Reasonable
Investment Level
Unusable Data /
Wasted opportunity
18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Architecture and Design
Streaming, GPU/CPU, Distributed with a Shared State
19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Streaming
State
20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Streaming
Data Size
Memory
Data Size
Time/Cost
21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Incremental Training
2
3
1
2
22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Incremental Training
3
1
2
23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
GPU/CPU
GPU State
24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Distributed
GPU State
GPU State
GPU State
25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Shared State
GPU
GPU
GPU Local
State
Shared
State
Local
State
Local
State
26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Cost vs. Time vs. Accuracy
$$$$
$$$
$$
$
Minutes Hours Days Weeks Months
27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
State Model
GPU State
28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Model Selection
1
1
29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Model Selection
1
30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Abstraction and Containerization
def initialize(...)
def update(...)
def finalize(...)
31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Production Readiness
Data/Model Size
Investment Reasonable
Investment Level
No unusable Data /
No wasted opportunity
32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Amazon SageMaker Algorithms
• DeepAR
• Factorization Machines
• Gradient Boosted Trees (XGBoost)
• Image Classification (ResNet)
• K-Means Clustering
• Latent Dirichlet Allocation (LDA)
• Linear Learner Classification and
Regression
• Neural Topic Modeling (NTM)
• Principal Components Analysis
(PCA)
• Random Cut Forest
• Seq2Seq
33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Linear Learner
Regression:
Estimate a real valued function
Binary Classification:
Predict a 0/1 class
˜y = hw, xi + t ˜y =
(
1 if hw, xi > t
0 else
34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
w1 = minw
X
i
L1(wT
xi, yi) + ↵1|w|1 + 1|w|2
wk = minw
X
i
Lk(wT
xi, yi) + ↵k|w|1 + k|w|2
Linear Learner
Train
Fit thresholds
and select
Select model with best validation performance
>8x speedup over naïve parallel training!
...
...
...
...
t = min
t0
L(˜y, y) ˜y =
(
1 if wT
i x > t0
0 else
35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Linear Learner
Regression (mean squared error)
SageMaker Other
1.02 1.06
1.09 1.02
0.332 0.183
0.086 0.129
83.3 84.5
Classification (F1 Score)
SageMaker Other
0.980 0.981
0.870 0.930
0.997 0.997
0.978 0.964
0.914 0.859
0.470 0.472
0.903 0.908
0.508 0.508
30GB datasets for web-spam and web-url classification
36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
K-Means Clustering
xi
1
n
X
i
min
j
kxi µjk2
µj
kxi µjk
38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
K-Means Clustering
Method Accurate? Passes Efficient
Tuning
Comments
Lloyds [1] Yes* 5-10 No
K-Means ++ [2] Yes k+5 to k+10 No scikit-learn
K-Means|| [3] Yes 7-12 No spark.ml
Online [4] No 1 No
Streaming [5,6] No 1 No Impractical
Webscale [7] No 1 No spark streaming
Coresets [8] No 1 Yes Impractical
SageMaker Yes 1 Yes
[1] Lloyd, IEEE TIT, 1982
[2] Arthur et. al. ACM-SIAM, 2007
[3] Bahmani et. al., VLDB, 2012
[4] Liberty et. al., 2015
[5] Shindler et. al, NIPS, 2011
[6] Guha et. al, IEEE Trans. Knowl. Data Eng. 2003
[7] Sculley, WWW, 2010
[8] Feldman et. al.
39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Principal Component Analysis (PCA)
kxi P(xi)k
P(xi)
xi
X
i
kxi P(xi)k2
41. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Principal Component Analysis (PCA)
42. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Neural Topic Modeling
Encoder: feedforward net
Input term counts vector
µ
z
Document
Posterior
Sampled Document
Representation
Decoder:
Softmax
Output term counts vector
Perplexity vs. Number of Topic
43. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
DeepAR –time series forecasting
Mean absolute
percentage error
P90 Loss
DeepAR R DeepAR R
traffic
Hourly occupancy rate of 963
bay area freeways
0.14 0.27 0.13 0.24
electricity
Electricity use of 370
homes over time
0.07 0.11 0.08 0.09
pageviews
Page view hits
of websites
10k 0.32 0.32 0.44 0.31
180k 0.32 0.34 0.29 NA
One hour on p2.xlarge, $1
zi,t 2, xi,t 1 zi,t 1, xi,t zi,t, xi,t+1
hi,t 1 hi,t hi,t+1
`(zi,t 1|✓i,t 1) `(zi,t|✓i,t) `(zi,t+1|✓i,t+1)
zi,t 1 zi,t zi,t+1
Input
Network
44. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Using AmazonSageMaker Algorithms
Command Line
SageMaker Notebooks
Amazon EMR
45. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Input Data
>> aws --profile <profile>
--region us-west-2
sm create-training-job
--training-job-name kmeans-demo
--algorithm-specification TrainingImage=0123456789.dkr.ecr.us-east-
1.amazonaws.com/kmeanswebscale:latest,TrainingInputMode=File
--role-arn "arn:aws:iam::0123456789:role/demo"
--input-data-config '{"ChannelName": "train", "DataSource":
{"S3DataSource":{"S3DataType": "S3Prefix", "S3Uri":
"s3://kmeans_demo/train", "S3DataDistributionType":
"FullyReplicated"}}, "CompressionType": "None", "RecordWrapperType": "None"}'
--output-data-config S3OutputPath=s3://kmeans_demo/output
--resource-config InstanceCount=2,InstanceType=c4.8xlarge,VolumeSizeInGB=50
--stopping-condition MaxRuntimeInHours=1
From Command Line
Hardware
46. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
From Amazon SageMaker Notebooks
Parameters
Hardware
Start Training
Host model
47. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
From Amazon EMR
Start Training
Parameters
Hardware
Apply Model
48. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Questions?
49. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Pop-up Loft
aws.amazon.com/activate
Everything and Anything Startups
Need to Get Started on AWS