2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Machine Learning at the Edge for
Industrial Applications
Richard Elberger
Global Partner Solutions Architect, IoT
Amazon Web Services, Partner Network

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
Tenets
Industrial IoT architecture
AIoT lifecycle – a four-part story

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Effectively maintain the system over its lifecycle

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Exploit system cost effectiveness via new intelligence

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Better decisions through central dashboards and
monitoring

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Derive value through new capabilities

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AWS IoT
Greengrass
Amazon
FreeRTOS
Amazon
FreeRTOS
Amazon
FreeRTOS
Amazon
FreeRTOS
Amazon
FreeRTOS

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Industrial
Control
Fieldbus

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AIoT – Machine Learning at the Edge
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
Train in the cloud and infer at the edge
AWS IoT
Core
AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon
EMR
Amazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground Truth
Apache
MXNet on
AWS
AWS Deep
Learning
AMIs
AWS
Snowmobile
AWS IoT
Greengrass
Amazon
FreeRTOS
AWS IoT
SiteWise
bespoke
applications

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AIoT – Machine Learning at the Edge
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
Train in the cloud and infer at the edge
AWS IoT
Core
AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon
EMR
Amazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground Truth
XILINX
DNNDK
AWS
Snowmobile
AWS IoT
Greengrass
UltraScale+
(DPU)
Amazon
FreeRTOS
Zynq7000
(DPU)

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AIoT – Machine Learning at the Edge
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
Train in the cloud and infer at the edge
AWS IoT
Core
AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon
EMR
Amazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground Truth
Apache
MXNet on
AWS
AWS Deep
Learning
AMIs
AWS
Snowmobile
AWS IoT
Greengrass
Amazon
FreeRTOS
AWS IoT
SiteWise
bespoke
applications

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
In the beginning

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Judgment on how to bring in data

19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AIoT – Machine Learning at the Edge
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
Train in the cloud and infer at the edge
AWS IoT
Core
AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon
EMR
Amazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground Truth
Apache
MXNet on
AWS
AWS Deep
Learning
AMIs
AWS
Snowmobile
AWS IoT
Greengrass
Amazon
FreeRTOS
AWS IoT
SiteWise
bespoke
applications

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Ingesting and methodically curating

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Iteratively evaluating and refining
Where data science meets art
annotation
cleansing
data types
re-process
raw data

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AIoT – Machine Learning at the Edge
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
Train in the cloud and infer at the edge
AWS IoT
Core
AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon
EMR
Amazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground Truth
Apache
MXNet on
AWS
AWS Deep
Learning
AMIs
AWS
Snowmobile
AWS IoT
Greengrass
Amazon
FreeRTOS
AWS IoT
SiteWise
bespoke
applications

25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
FRAMEWORKS
ML Frameworks +
Infrastructure
ML Services
AI Services
INTERFACES INFRASTRUCTURE
Amazon
SageMaker
Amazon
Transcribe
Amazon
Polly
Amazon
Lex
CHATBOTS
Amazon
Rekognition
Image
Amazon
Rekognition
Video
VISION SPEECH
Amazon
Comprehend
Amazon
Translate
LANGUAGES
P3 P3dn C5 C5n Elastic inference Inferentia AWS Greengrass
Ground Truth Notebooks Algorithms + Marketplace RL Training Optimization Deployment Hosting
AWS Confidential - Do not Distribute

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Deep learning frameworks and toolchains

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Feeding the art: data sets

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Training job: compilation

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Staging

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AIoT – Machine Learning at the Edge
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
Train in the cloud and infer at the edge
AWS IoT
Core
AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon
EMR
Amazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground Truth
Apache
MXNet on
AWS
AWS Deep
Learning
AMIs
AWS
Snowmobile
AWS IoT
Greengrass
Amazon
FreeRTOS
AWS IoT
SiteWise
bespoke
applications

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Mixed criticality system – block diagram

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Mixed criticality system – software placement
PS PL
AWS IoT
Greengrass

34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Mixed criticality system – pin wiring
PS PL

35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Updating the model

36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
InferenceHandler
m2m
PS
PL
AWS IoT
Greengrass

37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Device level dependencies
/dev/uio0
/dev/uio1
/dev/i2c-0
/dev/i2c-1
/dev/mem
InferenceHandler
m2m
PS
PL
AWS IoT
Greengrass

38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reporting inference results
ImageUploadHandler
m2m
PS
PL
AWS IoT
Greengrass

39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Staging data for building up the data lake
ImageStageHandler
m2m
PS
PL
AWS IoT
Greengrass

40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Recap
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
AWS IoT
Core
AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon
EMR
Amazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground Truth
Apache
MXNet on
AWS
AWS Deep
Learning
AMIs
AWS
Snowmobile
AWS IoT
Greengrass
Amazon
FreeRTOS
AWS IoT
SiteWise
bespoke
applications

42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS DeepLens
The world’s first deep learning-
enabled video camera for
developers
• A new way to learn ML though sample projects, with practical,
hands-on examples
• Run deep learning models locally on the camera to recognize
or classify without streaming to the cloud
• Easy to customize and fully programmable using AWS
Lambda
• Integrated with Amazon SageMaker for custom model
deployment
• Runs on any deep learning framework, including Apache
MXNet, TensorFlow, and Caffe.
Available now on amazon.com for $249

44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
GET STARTED WITH SAMPLE PROJECTS
ARTISTIC STYLE TRANSFER
OBJECT DETECTION FACE DETECTIONHOT DOG / NOT HOT DOG
CAT VS. DOG
ACTIVITY DETECTION
Or build custom deep learning models in the cloud using
Amazon SageMaker
HEAD POSE DETECTION

45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Intel Atom® Processor
• Gen9 graphics
• Ubuntu OS- 16.04 LTS
• 100 GFLOPS performance
• Dual band Wi-Fi
• 8 GB RAM
• 16 GB storage (eMMC)
• 32 GB SD card
• 4 MP camera with MJPEG
• H.264 encoding at 1080p resolution
• 2 USB ports
• Micro HDMI
• Audio out
• AWS Greengrass preconfigured
• clDNN-optimized for MXNet
• Key Differentiators/Technologies
• Intel cLDNN Library optimized for MXNet
• Intel Deep Learning Deployment Toolkit
AWS DeepLens Specifications

46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS DEEPLENS ARCHITECTURE
Video out
Data out
I N F E R E N C E
D E P L O Y P R O J E C T S
Manage device
Security
Console Project
Management
AWS Cloud
Intel: Model Optimizer
cIDNN and Driver

47. Thank you!
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Richard Elberger
Global Partner Solutions Architect - IoT
https://github.com/rpcme/
https://www.linkedin.com/in/richardelberger/
https://twitter.com/richardelberger