Presentation at International Advanced School on Knowledge Co-creation and Service Innovation 2012, Japan Advanced Institute of Science and Technology, March 1

1. A PRESENTATION OF
A REAL-TIME ROBUST OBJECT
TRACKING
Prarinya Siritanawan (SIIT)
Toshiaki Kondo (SIIT), Kanokvate Tungpimolrut (NECTEC), Itsuo Kumazawa (Tokyo Tech)
Master of Information and Communication Technology for Embedded System
Sirindhorn International Institute of Technology
Presentation at International Advanced School on Knowledge Co-creation and Service Innovation 2012,
Japan Advanced Institute of Science and Technology, March 1 1

2. OUTLINE
• Introduction
• Hamming Distance based Gradient
Orientation Pattern Matching
• Experimental Results
• Conclusion
• Question and Answer
2

3. INTRODUCTION
“OBJECT TRACKING IS A DETERMINATION OF LOCATION,
PATH AND CHARACTERISTICS OF AN INTERESTED OBJECT”
Subhash Challa, Mark R. Morelande, Darko Musichki and Robin J. Evans, “Fundamentals of Object
Tracking”, Cambridge University Press, 2011

4. INTRODUCTION
Applications for object tracking
• Video surveillance
• Human-machine interface
• Robot control
• Air space monitoring
• Weather monitoring
• Cell biology
Subhash Challa, Mark R. Morelande, Darko Musichki and Robin J. Evans, ”Fundamentals of Object
Tracking”, Cambridge University Press, 2011 4

5. INTRODUCTION
Major problems of visual tracking are caused by
• Illumination change
• Occlusion We focus on these problems
• Computation time
• Scaling
• Rotation
• Focus
• Aperture
5

6. INTRODUCTION
• Typical visual tracking and motion estimation
techniques assume that lighting conditions are
constant and minimal occlusion.
• We proposed a new template matching technique.
6

7. INTRODUCTION
• Template matching is the intensity-based technique
for measuring the similarity between template and
corresponding block of image.
x
Popular similarity metrics
SAD I (i , j ) T (i , j )
Template y i j
2
SSD I (i , j ) T (i , j )
i j
Match
7
Sample frame

8. INTRODUCTION
• We obtain an array of SADs or SSDs after scanning
the template over the entire image.
Best matching
position
Fig. 1. Inverted SSD
result.
8

9. INTRODUCTION
• However SADs and SSD are sensitive to changing
lighting conditions and occlusion. In order to
develop a method that can provide the robustness to
illumination change, a new template matching
technique is used.
• For illumination change problem, we introduced a
robust feature called Unit gradient vector (UGVs).
• To cope with the occlusion problem, we introduce
the Hamming distance as a new matching method
instead of SSD.
9

10. HAMMING DISTANCED BASED GRADIENT
ORIENTATION PATTERN MATCHING
10

11. HAMMING DISTANCE BASED GRADIENT
ORIENTATION PATTERN MATCHING
• Hamming distance based Gradient Orientation
Pattern Matching (P.Siritanawan & T.Kondo)
– Template matching based technique using Hamming
distance (HD) on Unit gradient vectors (UGVs).
Fig. 2. Intensity image. Fig. 3. Unit gradient vectors in x
and y direction
11

12. Sample Image Template
1st Derivative (Sobel Operator)
Step 1
gx1 gy1 gx2 gy2
Extract UGVs
feature Normalize
nx1 ny1 nx2 ny2
Threshold Threshold
Step 2 Absolute Diff. Absolute Diff.
Perform template
matching by using OR
Hamming distance Block at
position Sum
(x,y) Iterate (N-M-1) blocks
return [Best matching position (x,y)] 12

13. UNIT GRADIENT VECTORS
• UGV is a robust feature against Illumination changes.
Normal
condition
Intensity image Gradient vectors Unit gradient vectors
Lighting
change
condition
13

14. UNIT GRADIENT VECTORS
• The unit gradient vectors (UGVs) feature can be
extracted through the following normalized
equations
where Ix and Iy are gradient of intensities in x and y direction
is a small constant to prevent zero division
14

15. MATCHING METHOD
• We introduce Hamming Distance (HD),
0 0 1 1
0 0 1 1
= 0 0 1 1
0 0 0 1
1st Pattern 2nd Pattern HD(x,y) = 8
HD counts the number of pixels that are not match
15

16. MATCHING METHOD
• HD uses XOR but UGVs is not binary info.
• We need to transform the non-binary image
to be binary image using threshold absolute
difference function (O.Pele & M.Werman),
16

17. MATCHING METHOD
• Then the total distance of the block at position
(x,y) is given by
Fig. 4. Inverted HD
result.
17

18. MATCHING METHOD
Image Template Occluded Similarity Score
features Image metric
1 4 2 6 1 4 0 0 0 0 2 6
SAD
6 3 4 4 6 3 0 0 0 0 4 4
Intensities = 32
5 6 6 2 5 6 0 0 0 0 6 2
2 3 5 3 2 3 0 0 0 0 5 3
Unit 0 0 1 1
gradient 0 0 1 1 HD = 8
vectors 0 0 1 1
(UGV) 0 0 1 1
Pixelwise voting
18

19. EXPERIMENTAL RESULTS
19

20. DEMONSTRATION
20

21. EXPERIMENTAL RESULTS
Which is the ?
? Best matching
best matching ?
position ? ? peak found !!
Fig. 5. Tracking results under irregular lighting with
occlusion by (a) SSD on UGVs, (b) HD on UGVs, (c) and
(d) are the distributions of the corresponding similarity
measurements
21

22. CONCLUSION
• A novel pattern matching technique combines
the advantages of
– Unit gradient vectors (UGVs)
– Hamming distance metric (HD)
• UGV is a robust feature against the time-varying
lighting conditions.
• Compared with conventional matching with SAD
or SSD on intensity, HD yields better results in
partial occlusion scenarios. (60-70% covered).
• Efficient over the existing matching techniques on
both synthetic and real image sequences.
22

23. PUBLICATION
1. Wattanit Hotrakool, Prarinya Siritanawan, and Toshiaki Kondo, “Real-time
Gradient Orientation Pattern Matching”, International Conference on
Embedded System and Information Technology, Chaing Mai, Thailand, 2010
2. Wattanit Hotrakool, Prarinya Siritanawan, and Toshiaki Kondo, “A Real-time Eye-
tracking Method using Time-varying Gradient Orientation Patterns”, In proc.
ECTI-CON, Thailand, 2010
3. Prarinya Siritanawan and Toshiaki Kondo, “Hamming Distance based Gradient
Orientation Pattern Matching”, In proc. International Symposium of Artificial life
and Robotics 17th, Chaing Mai, Oita, Japan, January 2012
4. Prarinya Siritanawan, Toshiaki Kondo, Kanokvate Tungpimolrut, Itsuo Kumazawa,
“A visual tracking method using the Hamming distance”, In proc. International
Conference on Information and Communication Technology for Embedded
System 3rd, Bangkok, Thailand, March 2012
23

24. ACKNOWLEDGEMENT
This research is supported by
• National Research University Project of Thailand,
Office of Higher Education Commission
• Sirindhorn International Institute of Technology
(SIIT)
• Thailand Advanced Institute of Science and
Technology (TAIST)
• Tokyo Institute of Technology
• National Electronics and Computer Technology
Center (NECTEC)
24

25. THANK YOU FOR YOUR ATTENTION
QUESTION AND ANSWER
25