Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Globant - Amazon recognition workshop - 2018
1. Javier Cristancho – PSA AWS
Marzo - 2018
Amazon Rekognition
Deep learning-based image recognition service
2. Images – Universal, Ubiquitous, & Essential
There are 3,700,000,000 internet users in 2017
1,200,000,000 photos will be taken in 2017 (9% YoY Growth)
Source: InfoTrends Worldwide
3. Amazon Rekognition
Extract rich metadata from visual content
Object and Scene
Detection
Activity Detection
Facial
Analysis
Face
Comparison
Facial
Recognition
Person Tracking
Real time live Stream
Celebrity
Recognition
Image
Moderation
5. Object & Scene Detection
Object and scene detection makes it easy for you to add features that search,
filter, and curate large image libraries.
Identify objects and scenes and provide confidence scores
DetectLabels
Flower
Arrangement
Chair
Coffee Table
Living Room Indoors
Furniture
Cushion
Vase
Maple
Villa
Plant
Garden
Water
Swimming Pool
Tree
Potted Plant
Backyard
Patio
6. Facial Analysis
Analyze facial characteristics in multiple dimensions
DetectFaces
Image Quality
Facial Landmarks
Demographic Data Emotion Expressed
General Attributes
Facial Pose
Brightness 23.6%
Sharpness 99.9%
EyeLeft,EyeRight,Nose
RightPupil,LeftPupil
MouthRight,LeftEyeBrowUp
Bounding Box...
Age Range 29-45
Gender:Male 96.5%
Happy 83.8%
Surprised 0.65%
Smile:True 23.6%
EyesOpen:True 99.8%
Beard:True 99.5%
Mustache:True 99.9%...
Pitch 1.446
Roll 5.725
Yaw 4.383
7. Face Comparison
Measure the likelihood that faces are of the same person
Similarity 93% Similarity 0%
CompareFaces
9. Celebrity Recognition & Image Moderation
Newly released Rekognition features
Detect explicit and suggestive contentRecognize thousands of famous individuals
DetectModerationLabelsRecognizeCelebrities
11. What Can You Do with Amazon Rekognition?
• Search for people, objects, scenes, and concepts
across millions of images
• Filter inappropriate or specific content
• Verify identities by matching against reference faces
• Recognize individuals by matching faces to a collection
• Analyze user traffic hotspots and journey paths by
demographics and sentiment
12. Sentiment Analysis - Use Case
(Retail – In-store and Online)
Demographic and Sentiment Analysis
Female
Happy
Smiling
Male
No Facial Hair
Happy
Female
Sad
No Eyeglasses
13. Sentiment Analysis
Amazon RedshiftAmazon Quicksight
Live Subject In-Store Camera Application
Amazon S3
Analyze Faces
Shoppers enter and
browse in retail store
In-store cameras capture
live images of shoppers
A Lambda function is triggered
and calls Rekognition Rekognition analyzes the image and returns
facial attributes detected, which include
emotion and demographic detail
Return data is normalized and
staged in S3 en route to Redshift
Marketing Reports
Periodic ingest of data into
Redshift
Regular analysis to identify trends in
demographic activity and in-store
sentiment over time
Trend reporting for retail store locations
14. Look Your Best All Day
Time for A New Look?
Facial Analysis - Use Case
(Targeted Marketing)
Demographic and Sentiment Analysis
PersonAPersonB
Sees
Sees
15. Facial Analysis - Use Case
(Targeted Marketing)
Demographic and Sentiment Analysis
demographic and
sentiment attributes
Look Your Best All Day
display ad image
Application
AMAZON
REDSHIFT
AMAZON
DYNAMODBAMAZON S3
log
demographic
profile updates
retrieve ad image
face image is collected
and analyzed AMAZON
REKOGNITION
DetectFaces
Store
M
etadata
16. Collections and Access Patterns
Logging - public events; visitor logs; digital libraries
• One large collection per event/time period
• Wide searches
Social Tagging - photo storage and sharing
• One collection per application user
• Automated friend tagging
Person Verification - employee gate check
• One collection for each person to be verified
• Detection of stolen/shared IDs
17. {
"FaceMatches": [
{"Face": {"BoundingB
"Height":
0.2683333456516266,
"Left":
0.5099999904632568,
"Top":
0.1783333271741867,
"Width":
0.17888888716697693},
"
{
"FaceMatches": [
{"Face": {"BoundingB
"Height":
0.2683333456516266,
"Left":
0.5099999904632568,
"Top":
0.1783333271741867,
"Width":
0.17888888716697693},
"
Rekognition APIs – Overview
Rekognition’s computer vision API operations can be grouped into
Non-storage API operations, and Storage-based API operations
CompareFaces
DetectFaces
DetectLabels
DetectModerationLabels
GetCelebrityInfo
RecognizeCelebrities
Non-storage API Operations
CreateCollection
DeleteCollection
DeleteFaces
IndexFaces
ListCollections
SearchFaces
SearchFacesByImage
Storage-based API Operations
ListFaces
18. Rekognition APIs – Advanced Usage
Decision trees and processing pipelines
Why?
• Many use cases require more than a single
operation to arrive at actionable data
How?
• S3 event notifications, Lambda, Step Functions
• DynamoDB for persistent pipeline storage
• Augmenting results with 3rd Party AI/ML
• OpenCV, MXNet, etc. on EC2 Spot, ECS, AI/ML AMI
Sample Use Cases
• Person of interest near a celebrity
• Multi-pass motion detection enhancement
• Subjects leaving a location without possessions
IndexFaces
DetectLabels
“person”
21. • Built in 3 weeks
• Indexed against 99,000 people
• Index created in one day
• Saved ~9,000 hours a year in
manual curation costs
• Live video with frame sampling
Automating Footage Tagging with
Amazon Rekognition
Previously, only about half of all footage was indexed due to the
immense time requirements required by manual processes
23. Quickly Identifying Persons of Interest
with Amazon Rekognition
• More than 300,000 mugshots indexed within 1-2 days
• Identification process went from days (and weeks), to seconds
– greatly increasing the ability for law enforcement to act quickly
• Within 1 week of going live, the application helped identify a
suspect in a case that had no leads, leading to an arrest
There was no software on the market that allows users to quickly search
hundreds of thousands of images using a face in another image
24. Quickly Identifying Persons of Interest
with Amazon Rekognition
Solution Architecture
MS SQL
Database
ColdFusion
Web Service
Mugshot
Bucket
T2.Micro
MySQL DB
instance
Amazon
Rekognition
Amazon
Cognito
iOSMobile
Client
client PHP
1
2
3
4
25. • Media and Entertainment
• Digital Asset Management
• Safety and Security
• Law Enforcement
• Consumer Electronics
• Influencer Marketing
• Consumer Storage
• Travel and Hospitality
• Digital Advertising
• eCommerce
• Education
Rekognition Customers
26. Rekognition - Summary
• Leverage Amazon internal experience with
AI, ML and Computer Vision
• Managed API service with embedded AI for
maximum accessibility and simplicity
• Integrates natively with other AWS Services
• Extensible by design
28. Face-Based User Verification
Authenticated User
Image Capture Application
Amazon S3
Compare Faces
If the similarity score is over 92%, the
application returns a green status. If not,
an alert is issued to security staff.
The application captures a live
image of each employee as
they scan their access card
Rekognition compares the live image
and the badge image – and returns
a similarity score
The application retrieves the
user’s badge from S3
Confirm user identities by comparing their live image with a reference image