This document provides a summary of new artificial intelligence and internet of things services from Amazon Web Services that were announced in January 2017. It discusses Amazon Rekognition for image and facial analysis, Amazon Lex for building conversational interfaces, and Amazon Polly for converting text to speech. It also covers AWS Greengrass for running AWS Lambda functions and messaging on devices and the AWS IoT Button Enterprise Program.

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Gavin Adams, Enterprise Solutions Architect
January 25th, 2017
New Services Recap
Artificial Intelligence & IoT Services
Rekognition, Lex, Polly, AWS Greengrass, and
AWS IoT Button Enterprise Program

2. 1,017 New
Services and
Features in 2016
Artificial Intelligence (AI) and
Internet of Things (IoT)

3. Internet of Things
• AWS IoT
• Redesigned Management Console
• AWs IoT Button Enterprise Program (limited preview)
• 2nd Generation AWS IoT Button (2x battery life)
• AWS Greengrass
• Software for Connected Devices
• Local compute, messaging, and data caching
• Run IoT applications seamlessly across the AWS Cloud and
local devices using AWS Lambda

4. AWS IoT Enterprise Button
Program

5. How it works Invoke a
Lambda
function
Put object in
an S3 bucket
Read from or
Write to
DynamoDB
SNS Topic
or Endpoint
Publish to a
Kinesis
stream
{
"serialNumber":
"G030JF0552849P63",
"batteryVoltage":
"1543mV",
"clickType":
”SINGLE|DOUBLE|LONG”
}

6. Simplest way to build End to End Solutions
Developers Enterprise Program

7. Enterprise Program
• Order IoT Buttons in bulk
• Custom branded label
• Pre-provisioned security
• Many use cases:
• Reorder
• Click to call
• Retail Operations

8. AWS Greengrass

9. Things
Sense
& Act
AWS IoT
Starting in the cloud
Action
Device
State
AWS Services
Applications
Intelligence
Insights &
Logic → Action
Cloud
Storage
& Compute
Authentication
& Authorization
Device
Gateway
Registry
AWS IoT API
Messages Messages

10. Messages Messages
Authentication
& Authorization
Device
Gateway
Action
Device
State
AWS Services
Applications
Registry
AWS IoT API
AWS IoT
Going to the edge
Introducing AWS Greengrass
Device
State
Action
Device
Gateway
Messages

11. Greengrass is:
• Local logic execution
• Local triggers (message and broker)
• Security and access
• State sync with the cloud
*Note: Greengrass is NOT Hardware
(You bring your own)
Sign up at: https://aws.amazon.com/greengrass
Many applications require
edge compute
• Local timely decision making
• Privacy/regulatory requirements
• Functionality without connectivity

12. Why this is important
Data processed
in the cloud
Data
processed
locally
Embedded
Developer
Cloud
Developer
Program devices with
modern languages,
deployment APIs and
workflows
Cloud-based
development that adds
value to data which
never reach the cloud

13. Greengrass Core (GGC)
The runtime responsible for
Lambda execution, messaging,
device shadows, security, and for
interacting directly with the cloud

14. Greengrass Core (GGC)
• Min Single-Core 1GHz
• Min 128MB RAM
• x86 and ARM
• Linux (Ubuntu or Amazon)

15. Greengrass Core (GGC)
The sky is the limit.
GGC takes advantage of your
device’s compute, memory,
storage, and peripherals

16. How to get started today
Sign up for limited preview
Order a Snowball v2
http://aws.amazon.com/Greengrass

17. Artificial Intelligence
• Rekognition
• Image Analysis
• Lex
• Build Conversational Interfaces with Voice and Text
• Polly
• Turn Text into Lifelike Speech
• MXNet
• Open source fully featured deep learning framework

18. Amazon Rekognition

19. Images – explosive growth trends
Source: InfoTrends Worldwide Consumer Photos Captured and Stored.
2013 -2017 prepared for Mylio.

20. Amazon Rekognition
Deep learning-based image recognition service
Search, verify, and organize millions of images
Object and Scene
Detection
Facial
Analysis
Face
Comparison
Facial
Recognition

21. Amazon Rekognition API
DetectLabels
{
"Confidence": 94.62968444824219,
"Name": "adventure"
},
{
"Confidence": 94.62968444824219,
"Name": "boat"
},
{
"Confidence": 94.62968444824219,
"Name": "rafting"
},
. . .

22. Amazon Rekognition API
[
{
"BoundingBox": {
"Height": 0.3449999988079071,
"Left": 0.09666666388511658,
"Top": 0.27166667580604553,
"Width": 0.23000000417232513
},
"Confidence": 100,
"Emotions": [
{"Confidence": 99.1335220336914,
"Type": "HAPPY" },
{"Confidence": 3.3275485038757324,
"Type": "CALM"},
{"Confidence": 0.31517744064331055,
"Type": "SAD"}
],
"Eyeglasses": {"Confidence": 99.8050537109375,
"Value": false},
"EyesOpen": {Confidence": 99.99979400634766,
"Value": true},
"Gender": {"Confidence": 100,
"Value": "Female”}
DetectFaces

23. Demographic Data
Facial Landmarks
Sentiment Expressed
Image Quality
Facial Analysis
Brightness: 25.84
Sharpness: 160
General Attributes

24. Amazon Rekognition API
CompareFaces
{
"FaceMatches": [
{"Face": {"BoundingBox": {
"Height": 0.2683333456516266,
"Left": 0.5099999904632568,
"Top": 0.1783333271741867,
"Width": 0.17888888716697693},
"Confidence": 99.99845123291016},
"Similarity": 96
},
{"Face": {"BoundingBox": {
"Height": 0.2383333295583725,
"Left": 0.6233333349227905,
"Top": 0.3016666769981384,
"Width": 0.15888889133930206},
"Confidence": 99.71249389648438},
"Similarity": 0
}
],
"SourceImageFace": {"BoundingBox": {
"Height": 0.23983436822891235,
"Left": 0.28333333134651184,
"Top": 0.351423978805542,
"Width": 0.1599999964237213},
"Confidence": 99.99344635009766}
}

25. Amazon Rekognition API
Face Recognition
Index and Search faces in a collection
Index
Search
Collection
IndexFaces
SearchFacesByImage

26. Amazon Rekognition API
Face
{
f7a3a278-2a59-5102-a549-a12ab1a8cae8,
02e56305-1579-5b39-ba57-9afb0fd8782d,
4c55926e-69b3-5c80-8c9b-78ea01d30690
}
SearchFacebyImage
Collection
Nearest neighbor
search
Face ID

27. Amazon Lex

28. Advent of Conversational Interactions
1st Gen:
Punch Cards & Memory Registers
2nd Gen:
Pointers & Sliders
3nd Gen:
Conversational Interfaces

29. Amazon Lex - Features
Text and speech language understanding: powered by the same technology as
Alexa
Enterprise SaaS connectors: connect to enterprise systems
Deployment to chat services
Designed for builders: efficient and intuitive tools to build
conversations; scales automatically
Versioning and alias support
@

30. Text and Speech Language Understanding
Speech
Recognition
Natural Language
Understanding
Powered by the same deep learning technology as Alexa

31. Versioning and Alias Support
AliasVersioning
• Supported for Intents, Slots, and Bots
• Enables multi-developer environment
• Rollback to previous versions
• Deploy different aliases to different platforms
• Run different stacks for dev, stage and prod environments
• Target different user groups with different aliases
v1 v2 v3 latest
v1 Dev
v2 Stage
v3 Prod

32. Amazon Lex – Use Cases
Informational Bots
Chatbots for everyday consumer requests
Application Bots
Build powerful interfaces to mobile applications
• News updates
• Weather information
• Game scores ….
• Book tickets
• Order food
• Manage bank accounts ….
Enterprise Productivity Bots
Streamline enterprise work activities and improve efficiencies
• Check sales numbers
• Marketing performance
• Inventory status ….
Internet of Things (IoT) Bots
Enable conversational interfaces for device interactions
• Wearables
• Appliances
• Auto ….

33. Lex Bot Structure
Utterances
Spoken or typed phrases that invoke
your intent
BookHotel
Intents
An Intent performs an action in
response to natural language user
input
Slots
Slots are input data required to fulfill
the intent
Fulfillment
Fulfillment mechanism for your intent

34. Register for the Preview @
https://aws.amazon.com/lex
Sign-up & whitelist
Build your first bot!
✔
✔
✔

35. Facial Analysis – Demo
(Large Group Queries)
Demographic and Feature Analysis
Amazon
DynamoDB
Amazon Lex
Amazon
Rekognition
Analyze
ImageAmazon API
Gateway
S3 Bucket
Image
Bot
Function

36. Amazon Polly

37. What is Amazon Polly
• A service that converts text into lifelike speech
• Offers 47 lifelike voices across 24 languages
• Low latency responses enable developers to build
real-time systems
• Developers can store, replay and distribute
generated speech

38. Amazon Polly: Quality
Natural sounding speech
A subjective measure of how close TTS output is to human speech.
Accurate text processing
Ability of the system to interpret common text formats such as abbreviations, numerical
sequences, homographs etc.
Today in Las Vegas, NV it's 54°F.
"We live for the music", live from the Madison Square Garden.
Highly intelligibile
A measure of how comprehensible speech is.
”Peter Piper picked a peck of pickled peppers.”

39. Amazon Polly features: Lexicons
Enables developers to customize the pronunciation of
words or phrases
My daughter’s name is Kaja.
<lexeme>
<grapheme>Kaja</grapheme>
<grapheme>kaja</grapheme>
<grapheme>KAJA</grapheme>
<phoneme>"kaI.@</phoneme>
</lexeme>

40. Goal: Convert text into intelligible, accurate, and natural speech
Challenges:
• Homographs: words written identically that have different
pronunciation
I live in Las Vegas vs This presentation broadcasts live from Las Vegas
• Text normalization: disambiguation of abbreviations, acronyms, units
‘St.’ expanded as ‘street’ or ‘saint’
• Conversion of text to phonemes (Grapheme-to-Phoneme) in
languages with complex mapping such as English e.g. tough,
through, though
• Foreign words (déjà vu), proper names (François Hollande), slang
(ASAP, LOL) etc.
Main Challenges of Text-to-Speech

41. My First Polly App
from boto3 import Session
from contextlib import closing
polly = Session().client("polly")
response = polly.synthesize_speech(
Text="Hello world!",
OutputFormat="mp3",
VoiceId="Joanna")
with closing(response["AudioStream"]) as stream:
with open("speech.mp3", "wb") as file:
file.write(stream.read())

42. Thank you!