Presentation held by Mr. Petre Lameski as a part of the - Cooperation between academia and ICT businesses Session at the 8th SEEITA and 7th MASIT Open Days Conference, 14th-15th October, 2010

1. Mobile robot localization
Petre Lameski, MSc
Andrea Kulakov, PhD
Faculty of Electrical Engineering and Information
Technologies
{lameski, kulak}@feit.ukim.edu.mk
8th SEEITA – 7th SEE ICT Forum Meeting & 7th MASIT Open Days Conference
14-15 October 2010, Ohrid www.seeita.org

2. Overview
• Motivation
• Saccades
• SURF (Speeded up robust features)
• Learning subsystem
• Recognition subsystem
• Test results
• Discussion
8th SEEITA – 7th SEE ICT Forum Meeting & 7th MASIT Open Days Conference
14-15 October 2010, Ohrid www.seeita.org

3. Motivation
• The human brain remembers objects not only
by the light influx into the retina, but by the
sensory motor interaction between human
movement (including eye movement) and the
environment. (Behaviorist approach)
• Using computer vision in mobile robot
localization to overcome problems with other
sensor types
8th SEEITA – 7th SEE ICT Forum Meeting & 7th MASIT Open Days Conference
14-15 October 2010, Ohrid www.seeita.org

4. Saccades
• Short or long movements of eyes during
scene/object observation
• Psychologists suggest that a human needs
around 200ms to observe and recognize
pattern or object
• Saccades move in an optimal way in order to
receive information from objects and scenes
8th SEEITA – 7th SEE ICT Forum Meeting & 7th MASIT Open Days Conference
14-15 October 2010, Ohrid www.seeita.org

5. Saccades
• T. Shipley and P. Kellman, From Fragments to Objects: Segmentation and Grouping in Vision
8th SEEITA – 7th SEE ICT Forum Meeting & 7th MASIT Open Days Conference
14-15 October 2010, Ohrid www.seeita.org

6. Scene observation with saccades
8th SEEITA – 7th SEE ICT Forum Meeting & 7th MASIT Open Days Conference
14-15 October 2010, Ohrid www.seeita.org

7. SURF – (Speeded up robust
features)
• Scale, rotation and translation invariant
descriptor for local image features.
• Fast detection of characteristic points.
• Good results in repetition of detected points
under different transformations and blur.
*Herbert Bay, Andreas Ess, Tinne Tuytelaars, Luc Van Gool, "SURF: Speeded Up Robust Features", Computer Vision and
Image Understanding (CVIU), Vol. 110, No. 3, pp. 346--359, 2008
8th SEEITA – 7th SEE ICT Forum Meeting & 7th MASIT Open Days Conference
14-15 October 2010, Ohrid www.seeita.org

8. System architecture
Perception Recognition
Recognized Action
system subsystem
Not recognized
Learning
Labeling
subsystem
8th SEEITA – 7th SEE ICT Forum Meeting & 7th MASIT Open Days Conference
14-15 October 2010, Ohrid www.seeita.org

9. Learning subsystem
Acqire image
Apply Label
Acqire SURF descriptor
with maximal hessian
from the central part of
Reached the attention window
Initialize new
Not reached
attention window
Check if the number of
saccades has reached the Memorize the
predefined value fixation point
Check if 90% of the image area is
checked by attention windows or a
certain predefined number of points is
reached
Memorize the geometric
characteristics between
this and the previous
Yes fixation point
Remember the
saccadic sequence No
Check if the SURF
Acquire SURF Move attention
descriptor has a maximal
descriptor from the window to a location
hessian value in the new
peripheral part of the with center in the
attention window and
attention window SURF descriptor
update if neccessary
8th SEEITA – 7th SEE ICT Forum Meeting & 7th MASIT Open Days Conference
14-15 October 2010, Ohrid www.seeita.org

10. Recognition subsystem
Initialize hypothesis Similar sequence
Acqire image for all memorized not found, go to
sequences learning
subsystem
Initialize attention Finished for all
window from acquired No memorized Yes
image sequences
Find similar descriptor Yes
in the attention window
No
with the saccadic
sequence in memory Finished
Saccadic
sequence
Move the saccadic
window of the
considered image in a
Similarity? No manner that imitates
the considered
saccadic sequence
from the memory
Yes
Increase the
probability for the
state represented by
the recognized
saccadic sequence
No
Confirm the
Probability larger
hypothesis about the Yes
than threshold
location
8th SEEITA – 7th SEE ICT Forum Meeting & 7th MASIT Open Days Conference
14-15 October 2010, Ohrid www.seeita.org

11. Results
Table 1. Results from direct descriptor matching without saccadic sequences
Percentage of Percentage of guessed rooms Responsiveness
matched descriptors
80% 56,25% 40%
60% 48,6% 60%
Table 2. Results from tests on the implemented system
Percentage of guessed rooms Responsiveness
54% 70%
70% 25%
8th SEEITA – 7th SEE ICT Forum Meeting & 7th MASIT Open Days Conference
14-15 October 2010, Ohrid www.seeita.org

12. Human response
8th SEEITA – 7th SEE ICT Forum Meeting & 7th MASIT Open Days Conference
14-15 October 2010, Ohrid www.seeita.org

13. Conclusion
• The obtained results from the localization
show that the comparison of the saccadic
sequences of the descriptors outperforms the
naïve descriptor matching of the images.
• Improvements are possible in all segments of
the proposed system especially in the data
organization and representation.
• The way the scene is observed influences the
precision of room localization.
8th SEEITA – 7th SEE ICT Forum Meeting & 7th MASIT Open Days Conference
14-15 October 2010, Ohrid www.seeita.org

14. Future work
• Use of saccades in face and object recognition
• Improvements of the time complexity of the
current approach
• Smart-phone implementation
• Commercial applications that use this
approach. (under development together with
NI TEKNA -Intelligent Technologies
(www.ni-tekna.com)
8th SEEITA – 7th SEE ICT Forum Meeting & 7th MASIT Open Days Conference
14-15 October 2010, Ohrid www.seeita.org