1. Gippsland School of Information Technology (GSIT)
Behaviour Recognition
Framework for Intelligent
Visual Surveillance
06 April 2009
www.monash.edu.au
2. Project Team
• Mahfuzul Haque, PhD student (2 yrs, 1 m)
• A/Prof. Manzur Murshed
• Dr. Manoranjan Paul
www.monash.edu.au
2
3. Project Motivation
“Behaviour Recognition Framework for
Intelligent Visual Surveillance”
Why “Intelligent” Surveillance?
Why “Behaviour Recognition”?
What type of “Behaviours”?
www.monash.edu.au
3
5. Too Many Cameras
Deployment of large number of surveillance cameras in recent years
London Heathrow airport has more than 5000 cameras!!
www.monash.edu.au
5
6. Behind the Scene: Worried Human Monitor
Dependability on human monitors has increased.
Reliability on surveillance system has decreased.
www.monash.edu.au
6
7. Project Goals
Aiding human monitors by
automatic detection of
specific abnormal behaviors
Decreasing dependability on
human monitors
Improving reliability of
surveillance systems for
ensuring human security
www.monash.edu.au
7
9. Research Question
How to recognize specific group
behaviours from surveillance video
streams in real-time?
Research Area
- Computer Vision
- Application of Machine Learning
www.monash.edu.au
9
12. Behaviour Profile
Surveillance Video Stream (System Input)
Time
0
Unknown
20
Group
Appearing
60
Group
Appearing
140
320
Group
Merging
Group
Splitting
Behaviour Profile (System Output)
www.monash.edu.au
12
17. Background Modelling
How to extract the active regions from surveillance video stream?
Background Subtraction
Current frame
Challenges!!
=
Background
Moving foreground
www.monash.edu.au
17
29. Behaviour Classification
•
•
•
•
•
•
•
•
Individual classifiers for each behaviour
Supervised training
Feature ranking
Top 100 features from 270 features
Dimension reduction (PCA)
Max dimension 30
SVM classifier
Output: Behaviour Profile
www.monash.edu.au
29
30. Behaviour Classification
Experiments
GROUP FORMING
• Accuracy: 0.9767
• Top 3 features
• TIME(MAX)-VFD
• TIME(MAX)-AFD
• TIME(MAX) - TIME(MIN)-VFD
GROUP SPLITTING AND SPREADING
• Accuracy: 0.8488
• Top 3 features
• TIME(MAX)-VFD
• TIME(MIN)-ABBA
• TIME(MIN)-AFA
BLOCKED EXIT
• Accuracy: 0.9651
• Top 3 features
• TIME(MIN)-TFA
• MIN-MINAR
• TIME(MAX)-TFA
www.monash.edu.au
30
31. Summary: Framework Components
Background
Modelling
Frame Level Feature
Extraction
Temporal Feature
Extraction
Behaviour
Classification
• Multiple Background
Models
• Gaussian Mixture
Models (GMM)
• Unsupervised
• Output: Foreground
Region/Mask
• Feature Categories:
• Count
• Area
• Density
• Bounding Box
• Filling Ratio
• Aspect Ratio
• Output: 30 Frame Level
Features
• Fixed Length, Partially
Overlapped Sliding
Window
• Temporal Data
Smoothing –
Polynomial Curve
Fitting
• 9 Temporal Features for
Each Frame Level
Features
• Output: 270 Temporal
Features
• Individual Classifiers for
Each Behaviour
• Each Classifier is
Trained Using
Supervised Learning
• Feature Ranking
• Top 100 Features
• Dimension Reduction
(PCA)
• Max Dimension 30
• SVM classifier
• Output: Behaviour
Profile
www.monash.edu.au
31
32. Summary: Framework Components
Background
Modelling
Frame Level Feature
Extraction
Temporal Feature
Extraction
Behaviour
Classification
• Multiple Background
Models
• Gaussian Mixture
Models (GMM)
• Unsupervised
• Output: Foreground
Region/Mask
• Feature Categories:
• Count
• Area
• Density
• Bounding Box
• Filling Ratio
• Aspect Ratio
• Output: 30 Frame
Level Features
• Fixed Length, Partially
Overlapped Sliding
Window
• Temporal Data
Smoothing –
Polynomial Curve
Fitting
• 9 Temporal Features for
Each Frame Level
Features
• Output: 270 Temporal
Features
• Individual Classifiers for
Each Behaviour
• Each Classifier is
Trained Using
Supervised Learning
• Feature Ranking
• Top 100 Features
• Dimension Reduction
(PCA)
• Max Dimension 30
• SVM classifier
• Output: Behaviour
Profile
www.monash.edu.au
32
33. Research Challenges
• No tracking/trajectory
• Simple behaviours
– Group appear/disappear
– Group merge/split
• Panic driven behaviours
– Fire/Blocked exit
– Fighting/Shooting
• Context variation
– Speed
– Direction
– Object Resolution
www.monash.edu.au
33
35. Publications
1. Mahfuzul Haque, Manzur Murshed, and Manoranjan Paul, Improved
Gaussian Mixtures for Robust Object Detection by Adaptive MultiBackground Generation, International Conference on Pattern
Recognition (ICPR), Tampa, Florida, USA, 2008. (CORE A)
2. Mahfuzul Haque, Manzur Murshed, and Manoranjan Paul, A Hybrid
Object Detection Technique from Dynamic Background Using
Gaussian Mixture Models, IEEE International Workshop on Multimedia
Signal Processing (MMSP), Cairns, Australia, 2008. (CORE A)
3. Mahfuzul Haque, Manzur Murshed, and Manoranjan Paul, On Stable
Dynamic Background Generation Technique using Gaussian Mixture
Models for Robust Object Detection, IEEE International Conference On
Advanced Video and Signal Based Surveillance (AVSS), Santa Fe,
New Mexico, USA, 2008. (CORE A)
CORE - COmputing Research and Education Association
www.monash.edu.au
35
