1. Gait Recognition from Video
Mahfuzul Haque
1

2. Background

Biometric Recognition


Modes of Biometric Recognition



Biometric recognition or simply biometrics refers to the automatic
recognition of individuals based on their physiological and/or
behavioral characteristics.
Verification
Identification
Goal
To confirm an individual’s identity based on “who he is” rather than
“what he possesses” or “what he remembers”.
2

3. Various Biometric Characteristics














DNA
Ear (Cartilegenous tissue of the pinna are distintive)
Face
Facial, hand and hand vein infrared thermo gram
Fingerprint
Gait
Hand and finger geometry
Iris
Keystrokes
Odor
Palm print
Retinal Scan
Signature
Voice
3

4. Why Gait is getting more attention?



Non-contact human identification
Can be captured from a great distance
Great importance in security
4

5. Applications


Human recognition/identification
Activity recognition
5

6. Research approaches in recent works

Model based



Motion based




Extracted image feature mapped to a model
Computational cost high
Motion pattern converted to a compact representation
Lower complexity
Simpler implementation
Mixed approach

Combination of above two approaches
6

7. Available Databases



USF Human ID database
(http://www.gaitchallenge.org)
Southampton HiD database
(http://www.gait.ecs.soton.ac.uk)
CMU Mobo data set
7

8. Recent works
8

9. 1. An introduction to biometric recognition
Jain, A.K.; Ross, A.; Prabhakar, S.;
Circuits and Systems for Video Technology, IEEE Transactions on
Volume 14, Issue 1, Jan. 2004 Page(s):4 – 20
Comments:
This paper presents a brief overview of the field of biometrics and summarizes some of its advantages,
disadvantages, strengths, limitations and related privacy concerns. Following areas have been highlighted:
•Architecture of different types of biometric system and various modules.
•Modes of biometric system: identification and verification.
•Various biometric system errors
•Comparison of various biometrics
•Applications of biometric systems
•Advantages and disadvantages of biometrics
•Limitations of biometric systems
•Multimodal biometric systems (Fusion)
•Social acceptance and privacy issue
In general, it’s good paper presenting a concrete overview of biometric system and various biometric
characteristics.
9

10. 2. Fusion of static and dynamic body biometrics for gait recognition
Liang Wang; Huazhong Ning; Tieniu Tan; Weiming Hu;
Circuits and Systems for Video Technology, IEEE Transactions on
Volume 14, Issue 2, Feb. 2004 Page(s):149 - 158
Comments:
•Human recognition algorithm by combining static and dynamic body biometrics.
•Static features: body height, build
•Dynamic features: Joint angle trajectories of main limbs – how the static silhouette shape changes over time
•Both static and dynamic information may be independently used for recognition using the nearest exemplar
pattern classifier
Results:
Experiment on 20 subjects demonstrates the feasibility of the approach.
10

11. 3. Quantifying and recognizing human movement patterns from monocular video Imagespart I: a new framework for modeling human motion
Green, R.D.; Ling Guan;
Circuits and Systems for Video Technology, IEEE Transactions on
Volume 14, Issue 2, Feb. 2004 Page(s):179 – 190
Comments:
This paper presents a framework which forms a basis for the general biometric analysis of continuous human motion and demonstrated
through tracking and recognition of hundreds of skills.
Techniques
Computer vision-based framework
CHMR (Continuous Human Movement Recognition) framework
3D color clone-body-model is dynamically sized and texture mapped to each person for more robust tracking of edges and textured
regions during initialization phase.
Automatic initialization
Estimation of joint angles for the next frame using a forward smoothing particle filter
35 Dynemes: units of full-body movement skills defined.
CHMR uses multiple Hidden Markov Model (HMM) to infer human movement skill
Initialization phase required
Automated initialization assumes only one person is walking upright in front of a static background
Video->[Special Segmentation, feature extraction] -> motion vectors ->[Temporal Segmentation, hypothesis search]
Dyneme model: motion vector sequence -> dyneme
Skill model: dyneme sequence -> skill
Context model: skill pair or triplet
Activity model: Skill sequence -> activity
Results
Recognition was processed using the HMM Tool Kit (HTK )96.8% recognition accuracy on the training set and 95.5% recognition
accuracy on the independent test set.
No standard data set has been used
Tools and Terms
HTK – The HTK is a portable toolkit for building and manipulating hidden Markov models.
Gaussian prior
Future Work
Expanding dyneme model to improve discrimination
Expanding clone body model to include a complete hand-model
Use of multi-camera multimodal vision system to better disambiguate the body parts
11

12. 4. Quantifying and recognizing human movement patterns from monocular video imagespart II: applications to biometrics
Green, R.D.; Ling Guan;
Circuits and Systems for Video Technology, IEEE Transactions on
Volume 14, Issue 2, Feb. 2004 Page(s):191 – 198
Comments:
Presented as a continuation to part I of this series.
•Using the CHMR framework introduced in part I following applications have been demonstrated
•Biometric authentication of gait, Anthropometric data, Human activities and movement disorder
Techniques:
•Body part dimensions are quantified using the CHMR body model
•Gait signatures are then evaluated using motion vectors, temporally segmented by gait dynemes and projected
into a gait space for an Eigengait-based biometric authentication
•Left-right asymmetry of gait is also evaluated.
•CHMR activity model is used to identify various activities
•Movement disorders were evaluated by studying patients of Parkinsonism
Results:
•Anthropometric signature: 92%
•Gait Signature: 88%
•Fusion of both: 94%
•Activity: 0% error
•Parkinson’s Disease (PD): 95%
Future Works:
•Extending the models for loose clothing and carried items
•Increasing tracking stability by enhancing body models to include more degrees of freedom
•Improving the accuracy by increasing sample size
12

13. 6. The humanID gait challenge problem: data sets, performance, and analysis
Sarkar, S.; Phillips, P.J.; Liu, Z.; Vega, I.R.; Grother, P.; Bowyer, K.W.;
Pattern Analysis and Machine Intelligence, IEEE Transactions on
Volume 27, Issue 2, Feb. 2005 Page(s):162 – 177
Comments:
The paper presents a basis infrastructure for identification of people by analysis of gait patterns from video.
•The problem HumanID Gait Challenge has been introduced
•The challenge problem consists of a baseline algorithm, a set of 12 experiments and a large data set. Data
collected at University of Southern Florida in 2001.
•The dataset consists of 1870 sequences from 122 subjects spanning five covariates (1.2GB)
•The covariates are:
1. Shoe type
2. Carrying or not carrying a briefcase
3. Walking surface
4. Camera angle
5. Time
•All materials available at http://www.gaitchallenge.org
Techniques:
Estimating silhouettes by background subtraction and performing recognition by temporal correlation of
silhouettes.
Results:
•78% on easiest experiment, 3% on the hardest. Result benchmarked on the CMU Mobo data which is a
commonly used dataset for which performance has been reported in numerous papers.
•Tools and Terms:
•Gaussian Mixture Model (GMM)
•Mahalanobis Distance
•Expectation Maximization (EM)
•Gallery
•Probes/Signature
13
In general, a milestone contribution in the field of gait recognition. The infrastructure and data set provided by this
work has been used by other researchers afterwards.

14. 7. A video database of moving faces and people
O'Toole, A.J.; Harms, J.; Snow, S.L.; Hurst, D.R.; Pappas, M.R.; Ayyad, J.H.; Abdi, H.;Pattern
Analysis and Machine Intelligence, IEEE Transactions onVolume 27, Issue 5, May 2005
Page(s):812 – 816
Comments:
This paper describes a database of static images and video clips of human faces and people that is useful for
various researches.
•This work was supported by a grant from the Human ID project of DARPA/DOD
•Complete data sets for 284 subjects.
•Students from UTD participated as subjects.
•Gait videos: parallel and perpendicular
•160 GB HDD
14

15. 8. Matching shape sequences in video with applications in human movement analysis
Veeraraghavan A; Roy-Chowdhury, A.K.; Chellappa, R.;
Pattern Analysis and Machine Intelligence, IEEE Transactions on
Volume 27, Issue 12, Dec. 2005 Page(s):1896 – 1909
Comments:
•An approach for comparing two sequences of deforming shapes
•Two methods: parametric (AR-Auto Regressive, ARMA-Auto Regressive Moving Average) and non-parametric
(DTW – Dynamic Time Wrapping)
•Data set: USF Human ID and CMU
•Results/Findings: Role of shapes & kinematics in human movement analysis from video
•Kundall’s definition of shape is used for feature extraction
•Shape deformations of a person’s silhouette as a discriminating feature
15

16. 10. Human Gait Recognition With Matrix Representation
Xu, D.; Yan, S.; Tao, D.; Zhang, L.; Li, X.; Zhang, H.-J.;
Circuits and Systems for Video Technology, IEEE Transactions on
Volume 16, Issue 7, July 2006 Page(s):896 – 903
Comments:
Matrix representation based approach has been proposed for Human Gait recognition
Techniques:
•Binary silhouettes over one gait cycle are averaged
•Each gait video sequence, containing a number of gait cycles, is represented by a series of gray level averaged
images
•Pre-processing step: Each gait video sequence, containing a number of gait cycles, is represented by a series of
gray level averaged images. Then a matrix based unsupervised algorithm namely coupled subspace analysis
(CSA) is employed to remove noise and most representative information.
•Final step: a supervised algorithm namely discriminant analysis with tensor representation is applied to further
improve classification ability.
Results:
Demonstrates a much better gait recognition performance than USF HumanID gait database.
16

17. 11. A Bayesian Framework for Extracting Human Gait Using Strong Prior Knowledge
Ziheng Zhou; Prugel-Bennett, A.; Damper, R.I.;
Pattern Analysis and Machine Intelligence, IEEE Transactions on
Volume 28, Issue 11, Nov. 2006 Page(s):1738 - 1752
Comments:
This paper proposes a Bayesian framework for extracting human gait.
Techniques:
•Depends on strong prior knowledge and learning
•A consistent Bayesian framework has been proposed for introducing strong prior knowledge into the system
•Model considers both static and dynamic (time invariant/variant) parameters.
•Model is easily modified to cater situations such walkers wearing clothing that obscures the limbs
•Hidden Markov model is used to detect the phases of images in walking cycle.
•Dataset: Southampton Human Identification at a distance(HiD) Database has been used
Results:
•Results comparable with baseline algorithm
•Not every result is better than the baseline algorithm
This paper handles the situations “such walkers wearing clothing that obscures the limbs” which are not
addressed by previous works.
17

18. 12. Detection of Gait Characteristics for Scene Registration in Video Surveillance System
Havasi, L.; Szlvik, Z.; Szirnyi, T.;
Image Processing, IEEE Transactions on
Volume 16, Issue 2, Feb. 2007 Page(s):503 - 510
Comments:
•Presents a robust walk detection algorithm based on symmetry approach to extract gait characteristics from video
image sequence.
•Demonstrated application in image registration.
Techniques:
•Invariant and effective data representation in the Eigenwalk space, based on spline interpolation and a dimensionreduction technique.
Results:
•Reliable detection rate
18

19. Future research direction…





Improving performance and accuracy
Naturalistic context
Recognition with older database
..
..
19

20. Thank you
20