My Work Completion Seminar Presentation

1. Mahdi Babaei
1131600043
Research scholar – TM R&D Project
MASTER OF SCIENCE IN CREATIVE MULTIMEDIA
Supervisor: Assoc. Prof.Dr.Wong Chee Onn
Co-supervisor: Dr.Lim Yan Peng (Forest)
Work Completion Seminar

3. Introduction
Research Objectives
Research Questions
Literature review
Methodologies
Solution proposal
Results and Analysis
Conclusion

4.  With growth in number of innovation in new
devices on 1990th HCI came to daily life.
 From simple traditional GUI
 To

5. “How to optimize the best
combination of gesture
recognition methods in order to
have an efficient system (from
user view) while improving the
quality of interaction and human
factors, in a digital space?”

6.  To study on gesture recognition methods and
design a taxonomy to categorize them.
 To study on Digital space, content and projection
and design a space based on the knowledge.
 To study on virtual reality based interaction and
environment, pick factors and design a suitable
interaction based on that.
 To test interactive gesture recognition prototype
in virtual space with digital contents.
 To solve the disadvantages of proposed
combination method of gesture recognition in
virtual space with digital content.

8.  Gestures are expressive, meaningful body
motions involving physical movements of the
fingers, hands, arms, head, face, or body.
 Types based on(Billinghurst and Buxton 2011):
 Gestures in everyday World.
 Gestures only for interfaces.
 There are many taxonomies in this area like:
 (Kammer, Keck et al. 2010)
 (Karam 2005)

9.  We needed to design a new taxonomy of
gestures because:
 We need to go deep down in order to find the
exactly suitable gesture category that would
match the criteria and category can cover basics
only.
 Taxonomy provides vocabulary and Tree-Node
system and flexibility of adding new
vocabularies. On the other hand category can
offer a group of nodes with strict hierarchy.

10.  Based on (Barclay, Wei et al. 2011) :
 Fatigue (Barclay, Wei et al. 2011)
 Intuitiveness (Nielsen, Störring et al. 2004)
 Usability
 Learnability (Valkov, Steinicke et al. 2010)
 Easiness to navigate and orientate (Valkov, Steinicke et al.
2010)
 Naturalness and memorability (Lenman, Bretzner et al.
2002).
 Ergonomic

11.  Gesture time and duration(speed) (Barclay, Wei et
al. 2011
 Accuracy, Precision and error rate
 User Cooperation

12.  2D Interaction
 Two-Dimensional interaction in 3D world.
 Three-Dimensional interaction

13.  3D Interaction needs 3D space where user
would be able to have movements.
 It needs 3D contents (output) to let user
manipulate with them.
 It needs 3D hardware tools (input) to
detect depth value

14.  based on (Liu and Shrum 2002,Benyon 2010):
 Control desire (Burger and Cooper 1979, Liu and Shrum 2002)
 Accessibility
 Computer-mediated communication apprehension
(CMCA) (Liu and Shrum 2002)(The level of expertise in using computers)
 Usability: Based on (Standardization 1998, van Kuijk 2012):
 Effectiveness
 Efficiency
 Satisfaction
 Acceptability

15.  Based on (Hale and Stanney 2002) actions can be:
 Navigating through space.
 Specifying item of interest.
 Manipulating objects in the environment.
 Changing object values.
 Controlling virtual objects.
 Issuing task-specific commands.

16. Head-Mounted Displays

17.  Based on (Wilson and D’Cruz 2006) :
Influence of interaction on both sides.
User’s characteristics.
User’s needs.

19.  Virtual
 Physical(epistemic)
Two-degree freedom for 2D interaction
Multiple DOF for 2D interaction
Multiple DOF for 3D interaction
Gestures with tangible objects for 3D interaction
Gestures for Real-World physical object interaction
Paralinguistic
Linguistic
Act
Symbol
 Deictic
 Mimetic
 Gesticulations
 Metaphoric
 Affect displays
 Beat
 Referential
 Modelizing
Descriptive
Suggestive
Prompting
Emphatic
Side Effects of expressive behaviours
Mix-Communication Symbolic-
Interactive
Human Gestures
Symbolic (based on One-Way
Communication)
(Communicative or Semiotic)
Interactive (based on Two-Way
communication) or
Manipulative Communication
Emblems/Illustrators
Iconic
Regulators

20. Electromagnetic Acoustic Optical Mechanical
Advantage - - Fast upload rate -
Disadvantage
High inference with
magnetic field
Low rate target
positioning
Sight can obscured or
interfered
Limits user’s range of
motion
3D Model Based
Appearance-based
Volumetric Skeletal
Speed
Low
(complexity
of calculation)
High
( only key parameters
are analyzed)
Medium
(depends on algorithm)
Accuracy High High Medium
Processing time High Low Low
Points complicated 3D surfaces Skeleton Joints extraction Shape extraction

21. Three-Dimensional
Virtual Environment
Gesture Recognition
Interaction
Two-Dimensional
interaction in 3D
world
Two-Dimensional
3D- Hardware
3D- Space, 3D- Contents
System Control
Navigation
Selection
Object Manipulation Physical Movement
Manual viewport Manipulation
Steering
Target-based Travel Route planning

22. Accessibility
Computer-mediated
communication
apprehension
Usability
• Effectiveness
• Efficiency
• Satisfaction
AcceptabilityInfluence on
participants
Participant’s
Influence
User
Characteristics and
needs
Fatigue
Intuitiveness
Usability
• Learnability
• Easiness to
navigate and
orientate
• Naturalness and
memorability
• Ergonomic
Speed
Accuracy
User Cooperation
Virtual
Environment
Gesture
Recognition
Interaction
Control Desire

23. Gesture recognition
factors
Quantitative Factors Qualitative Factors
User expertise in using
computer systems
Usability
FatigueIntuitiveness
User cooperation
Speed
Accuracy
Acceptability
Learnability
Easiness to navigate and orientate
Naturalness and memorability
Ergonomic
Control Desire
Accessibility
Satisfaction
Efficiency
Effectiveness

24. Device Popularity Motor Driver SDK Image Quality Size Weight Power
Microsoft
Kinect
High Has HQ HQ Medium 12"x 3" x 2.5" 3.0 lb Ac + DC
ASUS Xtion /
PrimeSense
Carmine
Low No LQ LQ HQ 7" x 2" x 1.5" 0.5 lb DC-USB
 We choose to optimize Microsoft because:
 It has higher driver quality.
 It has software development kit.
 Popularity means easier access to research
resources.

25.  Can not track user’s eye and head
movements and rotations
 A combination of Microsoft Kinect as skeletal
detection device and an acceleration or
gyroscope data.

27. Head and
eye
Gestures
Body
Gestures
Kinect
Camera
Accelerometer
Digital Receiver
Analogue Receiver
Analogue to
Digital convertor
Antennas

28.  Yaw
 Pitch
 Roll
 Top/Left/
Bottom/ Right

29. Knee
Height
Time

32.  Before(Using Microsoft Kinect Only)
 After(Using proposed combination)
 Quantitative based on:
 Logical optimization.
 Optimization measurement based on results.
 Qualitative
 Questionnaire

33.  Speed
 16.6% improvement in coverage angle
-60
60
-90
90
-100
-50
0
50
100
Min Max
Before After
Degree

34. 527.64
315.32
418.37
0
100
200
300
400
500
Speed
AHRS Kinect Proposed method
Rotationspeedinone
second(DegreePerSecond)
99.38
59.39
78.80
0
20
40
60
80
100
Percentage
AHRS Kinect Proposed method
Percentageofsuccessful
recognizedgesture
 103.05(d/s)
improvement in
mean value
average of
rotation speed
(Degree Per
Second)
 19.41 %
improvement in
mean value of
successfully
recognized
gestures in a
second

35. 0.62
40.61
21.20
0
10
20
30
40
50
60
70
80
90
100
Error rate percentage in one second
AHRS Kinect Proposed Combination
ErrorratePercentage
19.41 % Reduction in mean value
average of error rate.

36. Questionnaire:
 100 participant.
 104 question (each factor 8 question).
 Same participant completed the same
questionnaire.
 Two steps: Before and After
 Likert 7 Scale
 Mean Value test

37. Reliability Statistics
Cronbach's Alpha N of Items
0.710 104
Reliability Statistics
Cronbach's Alpha N of Items
0.752 104
 Reliability Test

38. 4.92875
1.76
0
1
2
3
4
5
6
7
Mean Average Standard Deviation Average
Overal User Expertiese
Likert7Scale

39. 3.18
1.50
2.66
1.12
0
1
2
3
4
5
6
7
Mean Average Standard Deviation Average
Before After
Likert7Scale
 7.42 % Reduction in mean value average

40. 3.73
1.63
3.54
1.73
0
1
2
3
4
5
6
7
Mean Average Standard Deviation Average
Before After
Likert7Scale
 15.05 % Reduction in mean value average

41.  21.14% Improvement in Average of Mean value.
3.27
1.44
4.75
1.94
0
1
2
3
4
5
6
7
Mean Average Standard Deviation Average
Before After
Likert7Scale

42.  9.5% Improvement in average of mean value.
2.57
1.31
3.24
1.60
0
1
2
3
4
5
6
7
Mean Average Standard Deviation Average
Before After
Likert7Scale

43. 3.67
1.54
5.21
1.62
0
1
2
3
4
5
6
7
Mean Average Standard Deviation Average
Before After
22 % Increase in average of mean value
Likert7Scale

44. 3.12
1.47
4.32
1.32
0
1
2
3
4
5
6
7
Mean Average Standard Deviation Average
Before After
 17.14 % Increase in average of mean value
Likert7Scale

45. 3.02
1.13
4.10
1.25
0
1
2
3
4
5
6
7
Mean Average Standard Deviation Average
Before After
 15.42 % Increase in average of mean value
Likert7Scale

46. 2.60
1.13
3.14
1.14
0
1
2
3
4
5
6
7
Mean Average Standard Deviation Average
Before After
 7.71 % Increase in average of mean value
Likert7Scale

47. 3.98
1.51
4.76
1.81
0
1
2
3
4
5
6
7
Mean Average Standard Deviation Average
Before After
 11.42 % Increase in average of mean value
Likert7Scale

48. 3.50
1.57
5.42
1.69
0
1
2
3
4
5
6
7
Mean Average Standard Deviation Average
Before After
 27.42 % Increase in average of mean value
Likert7Scale

49.  27.85% improvement in average of mean value
3.23
1.10
5.18
1.44
0
1
2
3
4
5
6
7
Mean Average Standard Deviation Average
Before After
Likert7Scale

50.  10.28% improvement in average of mean value
4.62
1.60
5.34
1.25
0
1
2
3
4
5
6
7
Mean Average Standard Deviation Average
Before After
Likert7Scale

51. Usability
Sub-Factor
Mean Value Standard Deviation
Before After Before After
Learnability 3.27 4.75 1.44 1.94
Easiness to navigate and
orientate
3.67 5.21 1.54 1.62
Naturalness and memorability 3.02 4.10 1.13 1.25
Average 3.32 4.68 1.37 1.6
3.32
1.37
4.68
1.6
0
1
2
3
4
5
6
7
Before After
Mean Value Standard Deviation
 19.42% improvement in average of mean value
and more usable gesture recognition.
Likert7Scale

52. User
Cooperation
Sub-Factor
Mean Value Standard
Deviation
Before After Before After
User Control Desire over environment - 4.92 - 1.76
Accessibility 2.57 3.24 1.31 1.6
Satisfaction 3.98 4.76 1.51 1.81
Efficiency 3.5 5.42 1.57 1.69
Effectiveness 3.23 5.18 1.10 1.44
Average 3.32 4.704 1.37 1.66
19.77% improvement in average of mean
3.32
1.3725
4.704
1.66
0
1
2
3
4
5
6
7
Before After
Mean Value Standard Deviation
Likert7Scale

53. Factor Before After
Absolute value
of change
Percentage
Effectiveness 3.23 5.18 1.95 27.86
Efficiency 3.5 5.42 1.92 27.43
Easiness to navigate and orientate 3.67 5.21 1.54 22.00
Learnability 3.27 4.75 1.48 21.14
User Control Desire over environment 3.12 4.32 1.2 17.14
Naturalness and memorability 3.02 4.1 1.08 15.43
Satisfaction 3.98 4.76 0.78 11.14
Intuitiveness 4.62 5.34 0.72 10.29
Acceptability 2.57 3.24 0.67 9.57
Accessibility 2.59 3.14 0.55 7.86
Fatigue 3.18 2.66 0.52 7.43
Ergonomic and anxiety 3.73 3.54 0.19 2.71

54.  Wireless interactive gesture recognizer
device
 Gesture design contribution.
 High speed in tracking.
 High transmission speed.
 Designed Taxonomy and Framework.

55.  Qualitative factors general optimization
percentage average: 14%
 Quantitative factors general optimization
percentage average: 19.91%