VIDEO-BASED ACTION
RECOGNITION ON EDGE
01.04.2023

● How often a senior is treated in emergency room due to a fall?
2

● How often a senior is treated in emergency room due to a fall? - every 13 minutes
3

● How often a senior is treated in emergency room due to a fall? - every 13 minutes
● How much money do people spend due to falls?
4

● How often a senior is treated in emergency room due to a fall? - every 13 minutes
● How much money do people spend due to falls? - 69 billion $ annually
5

6
Structure
What
1
Why
2
How
3
Q&A
4

7
Who am I?
ANDRII
BOICHUK
Research Engineer
● Lecturer at NU “Lviv Polytechnic”
● 10+ year in IT, 5+ in research
● Tons of research activity

8
Who am I? (SQUAD Edition)
FOLLOW ON
FACEBOOK
FOLLOW ON
LINKEDIN
FOLLOW ON
INSTAGRAM
1250 +
Research Delivery
QA Embedded Hardware Mobile
Infrastructure Product Design
Kyiv
Lviv
Uzhhorod
STREAMS OF
TECHNOLOGIES
SQUAD – one of the biggest RnD center
in Ukraine. SQUAD – has some of the best
optical labs in Europe.

● 500+ студентів
● Вступний бал 190+
● 50% викладачів з галузі
9
Who am I? (NULP edition)
https://aidept.com.ua/

What?

● Action recognition
11
What is video-based action recognition on edge?

12
Action recognition
Action classification seeks to assign the correct label (e.g. “cooking,” “writing,” etc.)
to a given image or video.
Action localization, given a particular action and a video as input, seeks to identify
the correct location and timestamp in the video when the action is being performed.

● Action recognition
● Video based
13
What is video-based action recognition on edge?

● Action recognition
● Video based
14
What is video-based action recognition on edge?

● Action recognition
● Video based
● On edge
15
What is video-based action recognition on edge?

16
What is video-based action recognition on edge?
● Action recognition
● Video based
● On edge

Why?

18
Why action recognition?
● Saves lives

19
Saves lives

20
Why action recognition?
● Saves lives
● Saves lives and money

21
Saves lives and money

22
Why action recognition?
● Saves lives
● Saves lives and money
● Gives peace of mind

23
Gives peace of mind Most of the times

24
Why action recognition?
● Saves lives
● Saves lives and money
● Gives peace of mind
● Saves time

25
Saves time - sport highlights

26
Cloud problems
● Privacy

27
Cloud problems
● Privacy
● “bad internet connection”

28
Cloud problems
● Privacy
● “bad internet connection”
● Expensive

29
Cloud problems
● Privacy
● “bad internet connection”
● Expensive
● Streaming latency

30
Solution?
● Edge!

Why not?

● Harder
32

● Harder - HW, optimizations techniques and
combinations
33

● Harder
● Worse accuracy
34

● Harder
● Worse accuracy
● Uses additional resources which might be needed for
other tasks
35

Where?

37
Where?
● Surveillance cameras

38
Where

39
Where?
● Smart home cameras
● Fall detection

40
Where

How?

42
How
● Research

43
How
● Research - always check for SoTA - paperswithcode, github, CVPR,
EECV

44
How
● Research
● Hardware

45
How
● Research
● Hardware - know your limitations - CPU, GPU, FPGA or SoC?

46
How
● Research
● Hardware
● Optimizations

47
How
● Research
● Hardware
● Optimizations - quantization, pruning and hardware independent
techniques

48
How
● Research
● Hardware
● Optimizations

- CNN-based model
- GPU as hardware
- Pros: high throughput, high performance, no
optimizations needed
- Cons: memory latency, high energy consumption,
might be expensive
49
How

- Transformer-based model
- CPU as hardware
- Pros: low latency, universal, easy to use and optimize
- Cons: mid performance on other stats - mid energy
consumption, mid speed
50
How

51
How
● Research
● Hardware
● Optimizations
● Make your problem as narrow as possible

- Have the most basic baseline ready ASAP
- Chose target HW as soon as possible
- Go for SoTA
52
Tips

Q&A

Andrii Boichuk
Research engineer
Thank You!
SQUAD IN
FACEBOOK
SQUAD IN
LINKEDIN
SQUAD IN
INSTAGRAM