Collision Conference 2018 - AWS Developer Workshops Intro to AI & Machine Learning at Amazon: Ian Massingham

1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Ian Massingham | AWS Technical Evangelist @IanMmmm
aws.amazon.com/startups ianm@amazon.com
Intro to AI & Machine
Learning at Amazon

2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Fulfilment & logisticsAt Amazon,
we’ve been making
investments in ML for
the last 20 years…

4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Fulfilment & logistics
Search & discovery
At Amazon,
we’ve been making
investments in ML for
the last 20 years…

5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Fulfilment & logistics
Search & discovery
Existing products
At Amazon,
we’ve been making
investments in ML for
the last 20 years…

7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Fulfilment & logistics
Search & discovery
Existing products
New products
At Amazon,
we’ve been making
investments in ML for
the last 20 years…

11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Put machine learning in the
hands of every developer
and data scientist
ML @ AWS:
OUR MISSION

16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What are some of the
ML capabilities our customers are asking for?

17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vision
AWS ML Stack
Frameworks &
Infrastructure
GPU MobileCPU IoT (Greengrass)
Platform
Services
Application
Services Rekognition
Image
Speech
Polly
Language
Lex
TensorFlow GluonApache MXNet Cognitive Toolkit Caffe2 & Caffe PyTorch Keras
AWS Deep Learning AMI
Amazon Machine
Learning
Spark & EMR
Amazon Mechanical
Turk

18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vision
AWS ML Stack
Frameworks &
Infrastructure
AWS Deep Learning AMI
GPU
(P3 Instances)
MobileCPU IoT (Greengrass)
Platform
Services
Application
Services
Amazon
SageMaker
AWS
DeepLens
Rekognition
Image
Rekognition
Video
Speech
Polly Transcribe
Language
Translate ComprehendLex
Amazon Machine
Learning
Spark & EMR
Amazon Mechanical
Turk
TensorFlow GluonApache MXNet Cognitive Toolkit Caffe2 & Caffe PyTorch Keras

19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Customers Running Machine Learning On AWS Today

20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vision
Frameworks &
Infrastructure
GPU MobileCPU IoT (Greengrass)
Platform
Services
Application
Services Rekognition
Image
Speech
Polly
Language
Lex
TensorFlow GluonApache MXNet Cognitive Toolkit Caffe2 & Caffe PyTorch Keras
AWS Deep Learning AMI
Amazon Machine
Learning
Spark & EMR
Amazon Mechanical
Turk
AWS ML Stack

21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon EC2 P3 Instances
Up to eight NVIDIA Tesla V100 GPUs
1 PetaFLOP of computational performance – 14x better than P2
300 GB/s GPU-to-GPU communication (NVLink) – 9X better than P2
16GB GPU memory with 900 GB/sec peak GPU memory bandwidth
The fastest, most powerful GPU instances in the cloud

22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Deep Learning AMI
Get started quickly with easy-to-launch tutorials
Hassle-free setup and configuration
Pay only for what you use – no additional charge for the AMI
Accelerate your model training and deployment
Support for popular deep learning frameworks

23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Simplifies development of deep learning models
Provides greater flexibility in building neural networks
Offers high performance
Developed in collaboration with Microsoft
Available now in MXNet, coming soon to CNTK
Simple, Easy-to-Understand Code
Flexible, Imperative Structure
Dynamic Graphs
High Performance
Gluon
O C T O B E R 2 0 1 7

24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vision
Frameworks &
Interfaces
GPU (P3 Instances) MobileCPU IoT (Greengrass)
Platform
Services
Application
Services Rekognition
Image
Speech
Polly
Language
Lex
TensorFlow GluonApache MXNet Cognitive Toolkit Caffe2 & Caffe PyTorch Keras
AWS Deep Learning AMI
Amazon Machine
Learning
Spark & EMR
Amazon Mechanical
Turk
AWS ML Stack
Infrastructure

25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Machine learning process is hard…
Data wrangling
Setup and manage
Notebook environments
Get data to notebooks securely
Experimentation
Setup and manage clusters
Scale/distribute ML
algorithms
Deployment
Setup and manage
inference clusters
Manage and auto scale
inference APIs
Testing, versioning,
and monitoring
Fetch data
Clean & format
data
Train model
Evaluate
model
Integrate with
prod
Monitor/
debug/refresh
Prepare &
transform data

26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
6-18
months
…and time consuming
Fetch data
Clean & format
data
Prepare &
transform data
Train model
Evaluate
model
Integrate with
prod
Monitor/
debug/refresh

27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon SageMaker
A managed service that provides
the quickest and easiest way
for your data scientists and developers to get
ML models from idea to production.

28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
End-to-end machine learning platform
Zero setup Flexible model training Pay by the second
Amazon SageMaker
The quickest and easiest way to get ML models from idea to production

29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon SageMaker
BuildPre-built notebook
instances
Highly-optimized
machine learning
algorithms

30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Train
Amazon SageMaker
One-click training for
Ml, Dl, and custom
algorithms
Easier training with
hyperparameter
optimization
BuildPre-built notebook
instances
Highly-optimized
machine learning
algorithms

31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Deploy
Amazon SageMaker
Train
One-click training for
Ml, Dl, and custom
algorithms
Easier training with
hyperparameter
optimization
BuildPre-built notebook
instances
Highly-optimized
machine learning
algorithms
Deployment without
engineering effort
Fully-managed
hosting at scale

32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Streaming
datasets, for
cheaper training
Train faster,
in a single
pass
Greater reliability
on extremely
large datasets
Choice of
several ML
algorithms
Amazon SageMaker
10x better algorithms

33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Launch customers
Amazon SageMaker

34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vision
Frameworks &
Interfaces
GPU (P3 Instances) MobileCPU IoT (Greengrass)
Platform
Services
Application
Services Rekognition
Image
Speech
Polly
Language
Lex
TensorFlow GluonApache MXNet Cognitive Toolkit Caffe2 & Caffe PyTorch Keras
AWS Deep Learning AMI
AWS ML Stack
Infrastructure
Amazon
SageMaker
AWS
DeepLens
Amazon Machine
Learning
Spark & EMR
Amazon Mechanical
Turk

35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Object and scene
Detection
Facial
analysis
Face
comparison
Facial
recognition
Celebrity
recognition
Image
moderation
Amazon Rekognition Image
Deep learning-based image analysis service

36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Results:
| IT’S - 97% |
| MONDAY – 99% |
|but – 97% |keep – 96% |
| Smiling – 99% |
DetectText
Amazon Rekognition Image
N O V E M B E R 2 0 1 7
Text in image

37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
<0.5 second response time
Up to 10M faces
Enable Immediate response
Amazon Rekognition Image
N O V E M B E R 2 0 1 7
Real-time face recognition against tens of millions of faces

38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Rekognition Image
Customers

39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How can we apply these
powerful capabilities to video?

40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Manually
intensive
Slow and
error-prone
Expensive
Traditional solutions and limitations
Temporal
information lost
Motion
context
lost

41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Rekognition Video
Object and activity
detection
Person
tracking
Face
recognition
Real-time live
stream
Content
moderation
Celebrity
recognition
Video analysis

42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Rekognition Video
Stored Video
Amazon S3
Video Live Stream
Amazon Kinesis Video Stream
Media search index
Unsafe video detection
Investigative analysis
Public safety immediate response
Home monitoring
One solution for all

43. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Media & Entertainment Public Safety Smart Home
Search & Filter
Immediate Response
Investigative Analysis]
Monitoring
Amazon Rekognition Video
Primary use cases

44. Amazon confidential
Rekognition Video: Demo

45. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS ML Stack
Vision
Frameworks &
Interfaces
GPU (P3 Instances) MobileCPU IoT (Greengrass)
Platform
Services
Application
Services
Speech
Polly
Language
Lex
TensorFlow GluonApache MXNet Cognitive Toolkit Caffe2 & Caffe PyTorch Keras
AWS Deep Learning AMI
Infrastructure
Amazon
SageMaker
AWS
DeepLens
Amazon Machine
Learning
Spark & EMR
Amazon Mechanical
Turk
Rekognition
Image
Rekognition
Video

46. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Polly
Use Cases
Content Creation
Mobile & Desktop Applications
Internet of Things (IoT)
Education & E-Learning
Telephony
Game Development
Key Features
52 Voices across 25 languages
Lip-Syncing & Text Highlighting
Fine-grained Voice Control
Custom Vocabularies
Available in 14 AWS regions

47. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Polly Customers

48. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How do we enable analysis of all
audio content generated?

49. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Transcribe
Time
stamps
Support for
both regular &
telephony audio
Punctuation
&
formatting
S3
integration
Recognize
multiple
speakers
Custom
vocabulary
P R E V I E W
Automatic speech recognition
? !

50. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Transcribe
Use cases
Call
centers
Subtitles for
VOD
Transcribe
meetings
Broadcast
closed captions

51. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Transcribe
Launch customers

52. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS ML Stack
Vision
Frameworks &
Interfaces
GPU (P3 Instances) MobileCPU IoT (Greengrass)
Platform
Services
Application
Services
Speech
Polly
Language
Lex
TensorFlow GluonApache MXNet Cognitive Toolkit Caffe2 & Caffe PyTorch Keras
AWS Deep Learning AMI
Infrastructure
Amazon
SageMaker
AWS
DeepLens
Amazon Machine
Learning
Spark & EMR
Amazon Mechanical
Turk
Rekognition
Image
Rekognition
Video
Transcribe

53. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Lex
Scale
Business Logic
SecurityAnalytics
Text to Speech
Speech to Intent
End to
End
Native support &
maintains context
One-click deployment
Completely managed
service
Native integration
with AWS Lambda
Encrypted data
in transit & at
rest
Monitor and
improve
Amazon Polly
integrated into API
ASR + NLU integrated
into one API
Dialog Management Deployment

54. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Lex
Informational
Bots
Chatbots for everyday
consumer requests
Application
Bots
Build powerful
interfaces to mobile
applications
News updates
Weather information
Game scores ….
Book tickets
Order food
Manage bank accounts ….
Enterprise Productivity
Bots
Streamline enterprise
work activities and
improve efficiencies
Check sales numbers
Marketing performance
Inventory status ….
Internet of Things
(IoT) Bots
Enable conversational
interfaces for device
interactions
Wearables
Appliances
Auto ….
Contact Center
Bots
Chatbots for
customer service IVR
Account inquiries
Bill payment
Service update ….
Use cases

55. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Lex
Customers

56. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How do you communicate with your global
customers in the language of their choice?

57. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Neural machine translation
Amazon Translate
Real time
translation
Powered by
deep learning
12 language
pairs with more
to come
Language
detection

58. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
A “Pocket” Knife on amazon.de

59. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Really a very good Pocket knife. Had yesterday after breakfast something poppy
hang between the teeth. Because I once again found no toothpicks, I ordered mine
without further ADO this pocket knife. The integrated toothpick (Nr. 3) is very stable
and very quickly I could clean my teeth. Super! To the rest I can tell nothing
unfortunately because I him not be emergency.
A Happy Customer Wrote:

60. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
“Really a very good pocket knife. Had a little poppy between the teeth yesterday
after breakfast. Since I didn’t find a toothpick again, I ordered this pocket knife
shortly. The integrated toothpick (No.3) is very stable and I could clean my teeth
very quickly. Great! I can’t say anything about the rest because I don’t need it.”
The Neural Version

61. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Translate
Neural machine translation

62. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How do you extract insights
from unstructured text?

63. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Comprehend
Sentiment Entities LanguagesKey phrases Topic modeling
Powered By Deep Learning
Natural language processing

64. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon.com, Inc. is located in Seattle,
WA and was founded July 5th, 1994
by Jeff Bezos. Our customers love
buying everything from books to
blenders at great prices
Named Entities
- Amazon.com: Organization
- Seattle, WA: Location
- July 5th,1994: Date
- Jeff Bezos: Person
Keyphrases
- Our customers
- books
- blenders
- great prices
Sentiment
Positive
Language
English
Amazon Comprehend
Extract insight from text

65. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Fully managed
topic modeling
service
Amazon Comprehend
Topic modeling
Extract up to 100
topics from a corpus
of documents
Automatically
organizes documents
into the topics

66. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Voice of Customer
Analytics
Semantic
Search
Knowledge Management/
Discovery
Analyzing what customers are
saying about your brand,
products and services
Making search smarter by
searching on keyphrase,
sentiment and topic
Organizing documents,
categorizing by topic and
personalizing experiences
Amazon Comprehend
Use cases

67. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Comprehend
Launch customers

68. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vision
AWS ML Stack
Platform
Services
Application
Services
Amazon
SageMaker
AWS
DeepLens
Rekognition
Image
Rekognition
Video
Speech
Polly Transcribe
Language
Translate ComprehendLex
Amazon Machine
Learning
Spark & EMR
Amazon Mechanical
Turk
Frameworks &
Interfaces
GPU (P3 Instances) MobileCPU IoT (Greengrass)
TensorFlow GluonApache MXNet Cognitive Toolkit Caffe2 & Caffe PyTorch Keras
AWS Deep Learning AMI
Infrastructure

69. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Broadest & Easiest to Use ML Platform
Data Lake Storage
Amazon S3
Security
Access Control
Encryption
Compute
Powerful GPU & CPU Instances
AWS Lambda
Analytics
Amazon Athena
Amazon EMR
Amazon Redshift & Redshift Spectrum

70. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data Lake Storage
Amazon S3
Security
Access Control
Encryption
Compute
Powerful GPU & CPU Instances
AWS Lambda
Analytics
Amazon Athena
Amazon EMR
Amazon Redshift & Redshift Spectrum
Broadest & Easiest to Use ML Platform

71. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data Lake Storage
Amazon S3
Security
Access Control
Encryption
Compute
Powerful GPU & CPU Instances
AWS Lambda
Analytics
Amazon Athena
Amazon EMR
Amazon Redshift & Redshift Spectrum
Broadest & Easiest to Use ML Platform with the Most Customers

72. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Ian Massingham | AWS Technical Evangelist @IanMmmm
aws.amazon.com/startups
Thank you.
Any questions?