This document provides an overview of a project to develop an augmented reality face recognition application for mobile devices. It recaps the concept, which is to use a smartphone camera and face recognition to access extra information about people from social networks. It describes scenarios, technical details, an implementation using the MVC pattern in iOS with classes like a Facebook request wrapper and user model. It outlines a storyboard, shows problems encountered, and provides statistics on the project planning and hours spent on various aspects of the work. Evaluation results from user testing of paper and digital prototypes are also summarized.
4. Recap: Concept
Augmented reality face recognition for mobile devices…
Camera of smartphone will be
Real world augmented with extra used to accomplish goals
information, by using a camera -> Extra information on screen
Extra information will be
about people, by recognizing
their faces
Extra information will be
taken from social networks
… with social network information
4
5. Recap: Scenarios
1. Get contact info
2. Create a face based contact book
3. Quick access to slides/publications by
recognizing speaker
4. Quick access to social network information
5
6. Recap: Technical info
• iOS – Face detection
• iPhone
• Face.com – Free, SDK, private namespace,…
6
12. Paper prototyping: Iteration 2
• Positive: • Negative:
– Clean UI – Not enough home
– iOS style buttons
– The concept – Unclear contact icon
– Not him/her button!
12
13. Paper prototyping: Iteration 3
• Adapted to negative points
• Tested with prof Duval and 5 assistants
• Should improve:
– Incorrect button
– Still too much clicking to get somewhere =>
Tabbed Bar
– Delete person from history
13
19. Implementation:
CameraViewController
– Uses AVCaptureSession instead of
UIImagePickerController
– Linked with Face.com through API
– Uses iOS5 face detection to track faces
– Redraws facebox each frame
– Tried to be memory efficient
19
20. Implementation: UserModel
– Saves UID’s and names of friends
Reduces # requests
– Saves list of recognized persons
– Saves the settings
20
21. Implementation: RecognizedUser
– Saves UID and name of recognized person
– Saves profile picture
– Saves all other information received from
Facebook request
21
29. Evaluation: Iteration 1
– Tested with 7 smartphone users
– 4 already tested the paper prototype
– Current state of the app
– Focus on face recognition
– Results and comments were saturated fast
29
30. Evaluation: Iteration 1
– Think aloud
– Tasks
– Extra questions about satisfaction
– USE questionnaire
30
35. Todo
– Adapt to results evaluation
– Implement history and settings
– Training algorithm
– Multiple social networks
– Private namespace
Requires database
Which order???
35
39. Statistics
# Blog posts 21
# Comments on other blogs 10 (hcifetcher results: 3075)
# Tweets concerning thesis 81
Total # of hours worked 395
# of hours on literature study 20
# of hours on related work 10
# of hours on reports/blog posts 26
# of hours dedicated to other theses 18
# of hours on iOS learning 52
# of hours on design 35
# of hours on paper prototype testing 28
# of hours on paper prototype creating 35
and evaluating
# of hours on implementation 166
# of hours on digital prototype testing 5
39
