VEDLIoT Next Generation AIoT Applications. Micha vor dem Berge. VEDLIoT Conference Track co-located with IoT Tech Expo, Amsterdam, Netherlands, September 2023

1. Micha vor dem Berge (Head of R&D)
on behalf of all use case partners
VEDLIoT’s Next Generation
AIoT Applications

2. Big Picture

3. 3
Automotive AI
Motor Condition
Monitoring
Arc Fault Detection in DC
Power Distributions
Automatic Pedestrian
Emergency Breaking
Industrial IoT
Smart Home
Native
Human Interface

4. 4
Arc Fault Detection – Overview
Motivation
 Direct current (DC) systems are increasing in
both quantity and scale
 Series arc faults are difficult to detect and can
cause damage in DC systems
 AI-based solutions offer a general approach for
various circuit dynamics
Challenges
 Time sensitivity
 High accuracy requirement
 Adapting models to new systems

5. 5
Arc Fault Detection – Achievements
• Built test-bench for ML learning data
generation
• > 60000 times series datapoints
collected + labelled
• Trained Fully Connected Neural Network and
Convolutional Neural Network for classification
• Optimized AI model with EmbeDL SDK
• model size and runtime reduced by 70%
• Real-time data processing with DL algorithms
for arc detection within 1ms latency
• Model accuracy up to 99%
Arc fault detection based on real time sampled current data
Arc
Normal Normal

6. 6
Motor Condition Monitoring – Overview
Motivation
 Mount-on sensors for monitoring of big electrical
motors
 cooling system
 operational status
 mechanical condition
 On-site data processing, reduce network throughput
 Adaptive AI-based solution for different operational
environments
Challenges
 Power consumption limitation
 Hardware optimization for AI algorithms
 Integration of customizable sensors
Condition monitoring for direct driven motors. The usual
power range of such motors is between 5kW to 500kW

7. 7
Motor Condition Monitoring – Achievement
• > 18000 data points collected with
different cooling conditions
• Trained CNN model
• cooling system condition
classification
• based on temperature and vibration
data
• Model runtime 207us
• 2 convolutionary layers
+ 1 dense layer
• Developed custom SFD board with
integrated sensors and MAX78000
hardward AI accelerator
• Deployed Secure IoT Gateway
• Developed Augmented
Reality interface
Smart Field Device (SFD)
for Vibration, Temperature and Speed
Deep Learning Accelerator integrated
Max78000
integrated AI
accelerator
Temp
Sensor
External
interface
Power
Vibration
Sensor
Magnetic
Flux
Sensor
SRAM
Flash
JTag
VEDLIoT
MAX78000

8. 8
Smart Home – Overview
• Smart Mirror as a naturally integrated human interface
• Works like a conventional mirror
• Displays Information like weather, public transportation etc
• Interacts human-centric with residents
• Provides common features of smart home environments
• Integration of object, gesture, face, and speech recognition
• Using YoloV4(-tiny), YoloV7(-tiny), Siamese Network, FaceNet, etc
• Interaction via gestures and speech
• Data privacy highest priority!

9. 9
Smart Home – Achievements
• Integration of ROS 2 middleware
• Evaluation of different hardware setups
• Hailo-8, FPGA, Nvidia Jetson family etc.
• AI Optimizations with the EmbeDL tool
• Integration of trustworthy
speech recognition using encryption
• Creation of hand gesture dataset
• Face, person, and 38 gestures
• 73k images of 14 persons
• YoloV7-tiny map@.5 of 90%
• Coco Dataset for object detection

10. 10
Smart Home – Boosting Energy Efficiency
• Baseline: christmann t.RECS with 2x Nvidia Xavier AGX
• 16 FPS @ 150 Watt
• Currently: Nvidia AGX Orin + Hailo-8 AI accelerator
• 30 FPS @ 43 Watt
• Goal: Nvidia Orin NX + Hailo-8 AI accelerator
• 30 FPS @ 27 Watt

11. 11
 Detection and automatic emergency braking when pedestrians on road
 AEB chosen as reference point to analyse consequences of work distribution in automotive industry
 Optimization of pedestrian detection
 Minimization of false detection and false negatives
 Distributed AI processing via 5G
 Generation & optimization of DL model
 Generate DL learning data
 Design & train DL model
 Distribute work via cellular 5G
 Minimize response latency
 Safe and robust communication
Automatic Emergency Breaking – Overview
& Goals

12. 12
 Captured training dataset on airfield (60k images)
 Lots of different scenarios with pedestrians, objects,
weather, driving direction etc.
 Trained EfficientNet ML model
 Distributed ML model: car and 5G basestation
1. Everything in car
2. Everything remote in basestation
3. Hybrid
 Installed local 5G station for real-world tests
Automatic Emergency Breaking – Major
Highlights

13. 13

14. 14
Micha vor dem Berge
Head of R&D
[ Booth 232 ]