With the advancement of machine learning applications, new business opportunities are rapidly emerging in media. In this session, learn how the AWS Media2Cloud solution can save time and reduce costs through setting up a serverless end-to-end ingest workflow to move your video assets and associated metadata to the cloud. Gain insight into how to make those assets even more valuable by enabling searching and indexing on your video library, and learn how to use Amazon Transcribe and Amazon Translate to take your live-streaming workflows to the next level with expert instruction on how to enable automatically created multi-language subtitles.

1. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Increase the value of video using ML and AWS
media services
Bryan Samis
Solutions Architect
AWS
S V C 3 0 1

2. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Agenda
Why machine learning for media?
Use cases & relevant services
AWS solutions
• Media analysis solution
• Live captioning & translation solution

3. S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.

4. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Major shift in the Media &
Entertainment industry
Content is no longer king
The viewer is king
Direct-to-consumer is a business model
Success relies on a rich viewer experience
Rich viewer experiences rely on data and
analytics

5. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
The new reality
Viewer
expectation
Content discoverability, recommendations,
personalization
Business
opportunity
Learn about viewers, use the data to increase
engagement
Customer
obsession
Customer satisfaction increases brand value and
drives revenue

6. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Amazon Athena
Amazon EMR
Amazon Redshift
Amazon Elasticsearch Service
Amazon Kinesis
Amazon QuickSight
AnalyticsAmazon SageMaker
AWS Deep LearningAMIs
Amazon Rekognition
Amazon Lex
AWS DeepLens
Amazon Comprehend
Amazon Translate
Amazon Transcribe
Amazon Polly
Machine learning
AWS IoTCore
Amazon KinesisData Firehose
Amazon KinesisData Streams
Amazon KinesisVideo Streams
Real-time
data movement
AWS DirectConnect
AWS Snowball
AWS Snowmobile
AWS DatabaseMigrationService
On-premises
data movement
Data lake on AWS
Storage | Archival storage | Data Catalog
Incoming analytics data Existing asset metadata
Data lake + media assets = Content lake

7. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Modern data architecture for media enrichment
Insights toenhance viewer engagement, personalization, monetization
Improve UX / QoS
Recommend
Monetize
Infer
Store
& process
Content lake on AWS
Storage | Archival storage | Data Catalog
Analyze
Viewer data
IoT sensors
Social media
Video
Engage and personalize

8. S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.

9. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Sample ML use cases for media
Automate the creation of rich metadata index (object, scene, activity, faces etc.), extracted from audio visual
content and integrated into digital and media asset management systems
AWS services to build use case: Amazon Rekognition, Amazon SageMaker, Amazon Transcribe, Amazon Comprehend
M e d i a m e t a d a t a
t a g g i n g
Automated captions, transcription, and translation of audio content
AWS services to build use case: Amazon Transcribe, Amazon Translate, Amazon Comprehend
C l o s e d c a p t i o n i n g
Detect potentially inappropriate content to avoid compliance issues in global markets and increase brand safety
for advertisers
AWS services to build use case: Amazon Rekognition, Amazon Transcribe, Amazon SageMaker
A u t o m a t e d
c o m p l i a n c e m a r k i n g
Identify objects and emotion in content that enables users to integrate personalized ads into subscriber video
streams
AWS services to build use case: Spark/Amazon EMR, Amazon SageMaker, AWS Elemental MediaTailor
A d p e r s o n al i z a t i o n

10. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Sample ML use cases for media
Identify and path player activity in the field or identify the precise timecodes when actors enter and leave a
scene, both visually and in spoken dialog (off-screen voice)
AWS services to build use case: Amazon Rekognition, Amazon Transcribe
P l a y e r / A c t o r
i d e n t i f i c a t i o n &
a c t i v i t y p a t h i n g
Translate transcripts and metadata; improve localization workflows and search experience
AWS services to build use case: Amazon Translate
L a n g u a g e
t r a n s l a ti o n
Analyze disparate data and enable the consumer to make personalized content choices and the business to
predict behavior
AWS services to build use case: Amazon SageMaker, Amazon Personalize
C o n t e n t
r e c o m m e n d a ti o n s
Detect and pixelate faces captured incidentally to preserve the privacy of nonpersons of interest in news feeds
and security footage
AWS services to build use case: Amazon Rekognition, Amazon Transcribe, Amazon SageMaker
A u t o m a t e d
r e d a c t i o n

11. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Usecase: From manualtagging to AI-based tagging
Background
• Over 200,000 hoursof content
• Only half of contentis manually tagged
• How can we enrich our metadatain AWS?
• How can we unleash the value of content we already own
once in AWS?
Challenge
• Large-scale video library
• High accuracy required
• Limited budget
• Ability to extract from video
• Keep up with daily increase in content
Results
• Solution developed withinthree weeks
• Live video frame-based analysis
• Established,searchable baseline archive
• All contentis now tagged and indexed
• Over 99,000 faces indexed and searchable
• Saved ~9,000 hoursa year in manual curation costs

12. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Use case: Event broadcasting enrichment

13. S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.

14. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
The Amazon ML stack
M L f r a m e w o r k s &
i n f r a s tr u c t u r e
A I s e r v i c es
A m a z o n
R e k o g n i t i o n
I m a g e
A m a z o n
P o l l y
A m a z o n
T r a n s c r i b e
A m a z o n
T r a n s l a t e
A m a z o n
C o m p r e h e n d
& C o m p r e h e n d
M e d i c a l
A m a z o n
L e x
A m a z o n
R e k o g n i t i o n
V i d e o
Vi si on Speech Chatbots
A m a z o n S a g e M a k e r
B u i l d T r a i n
A m a z o n
F o r e c a s t
A m a z o n
T e x t r a c t
A m a z o n
P e r s o n a l i z e
D e p l o y
Pre-bui l t al gori thms & notebooks
Data l abel i ng (G r o u n d T r u t h )
One-cl i ck model trai ni ng & tuni ng
Opti mi zati on (N e o )
One-cl i ck depl oyment & hosti ng
M L s e r v i c e s
F r a m e w o r k s I n t e r f a c e s I n fr a s t r u c t u r e
A m a z o n
E C 2 P 3
& P 3 d n
A m a z o n
E C 2 C 5
F P G A s A W S
I o T
G r e e n g r a s s
A m a z o n
E l a s t i c
I n f e r e n c e
Hyperparameter opti mi zati on
Al go ri th ms & mo d el s ( A W S M a r k e t p l a c e
f o r m a c h i n e l e a r n i n g )
Language Forecasti ng Recommendati ons
No teb o o k h osti ng Rei nforcement l earni ng
Automati c scal i ng
Vi rtual pri vate cl oud
AWS Pri vateLi nk
El asti c Inference i ntegrati on

15. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Amazon Rekognition
Deep learning-based image analysis service
Object and scene
detection
Facial
analysis
Face search
& match
Text in imageContent moderation Celebrity recognition

16. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Amazon Rekognition Video
Analysis for live or stored video content

17. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Amazon Transcribe
Automatic speech recognition for stored or real-time audio
TimestampsSupport for
both regular &
telephony audio
Punctuation
& formatting
English,
Spanish,
French,Italian,
Brazilian
Portuguese
Recognize
multiple
speakers
Custom
dictionaries
? !

18. S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.

19. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
AWS media solutions
Accelerate revenue
Get started with your project quickly and start
monetizing your content
One-click launch
Launch a fully tested solution starter kit in your
AWS account with one click
Well architected
Rely on the architecture as it reflects AWS best
practices
Agility
Save the resources of developing a solution
from the beginning
Expand and add
Use the framework and available guides to expand the solution
starter kit based on your needs

20. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
AWS media solutions
http://aws.amazon.com/solutions/
Filter by: Industry > Media & Entertainment

21. S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.

22. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Media analysis solution
Label & face detection
Celebrity detection
Face search
Activity tracking
VoD subtitle generation
Key entity & phrase detection
Automated
metadata
generationA m a z o n
R e k o g n i t i o n
A m a z o n
R e k o g n i t i o n
V i d e o
A m a z o n
T r a n s c r i b e
A m a z o n
T r a n s l a t e
A m a z o n
C o m p r e h e n d
Transcription
Context
Amazon SageMaker + Additional solutionsfor media use cases

23. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Media analysis solution
Amazon Rekognition
Amazon Transcribe
Amazon Comprehend
Amazon S3 AWS Step
Functions
AWS Lambda
Amazon
Elasticsearch
Service
Image Video
AudioText
Amazon SageMaker

24. S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.

25. S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.

26. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Live subtitling solution
• Purpose built to help customers create live
streaming subtitling workflows on the AWS
Cloud
• Serverless architecture
• Uses new Amazon TranscribeStreaming to
extract speech for subtitles, then uses
Amazon Translateto convert into other
subtitle languages
• Deploy in minutes with a single click using an
AWS CloudFormation template
• Automatically spins up all resources needed
for the entire workflow
AWS Elemental
MediaLive
Lambda function
(caption creation)
Amazon
Transcribe
Amazon S3
bucket
Lambda function
(transcribe
streaming)
AWS Elemental
MediaPackage
Amazon
Translate
Manifests
Live channel
source
Amazon S3
bucket
Multi-language (optional)
PCM audio
Transcripts
Subtitles,manifests,&videofiles
PCM audio
Amazon
CloudFront

27. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Live subtitling solution
Ingest live video into AWS Elemental MediaLive
Amazon S3 events trigger subtitle creation
process with Amazon Transcribe Streaming
Translatespeech to other languages with
Amazon Translate
Live video with subtitles are ingested into AWS
MediaPackager
View your live stream in HLS, DASH, or
Microsoft Smooth Streaming
1
2
3
4
5
AWS Elemental
MediaLive
Lambda function
(caption creation)
Amazon
Transcribe
Amazon S3
bucket
Lambda function
(transcribe
streaming)
AWS Elemental
MediaPackage
Amazon
Translate
Manifests
Live channel
source
Amazon S3
bucket
Multi-language (optional)
PCM audio
Transcripts
Subtitles,manifests,&videofiles
PCM audio
Amazon
CloudFront
1 2
3
4
5

28. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Live subtitling solution
• Purpose built to help customers to create live
streaming subtitling workflows on the AWS
Cloud
• Serverless architecture
• Uses new Amazon TranscribeStreaming to
extract speech for subtitles then Amazon
Translateto convert into other subtitle
languages
• Deploy in minutes with a single click using an
AWS CloudFormation template
• Automatically spins up all resources needed
for the entire workflow
• Optionally add text to speech with Amazon
Polly or use Amazon Comprehend to
understand text sentiment
AWS Elemental
MediaLive
Lambda function
(caption creation)
Amazon
Transcribe
Amazon S3
bucket
Lambda function
(transcribe
streaming)
AWS Elemental
MediaPackage
Amazon
Translate
Manifests
Amazon S3
bucket
Multi-language (optional)
PCM audio
Transcripts
Subtitles,manifests,&videofiles
PCM audio
Amazon
CloudFront
Amazon
Polly
Amazon
Comprehend
Alternate audio(optional) Insights& sentiment(optional)

29. S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.

30. Thank you!
S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.
Bryan Samis
samisb@amazon.com

31. S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.