An Evaluation of the Leap Motion Depth Sensing Camera for Tracking Hand and Fingers Motion in Physical Therapy by Darryl Charles, Katy Pedlow, Suzanne McDonough, Ka Shek and Therese Charles

1. Close Range Depth Sensing
Cameras for Virtual Reality based
Hand Rehabilitation
Darryl Charles1, Katy Pedlow2, Suzanne McDonough2, Ka Shek3, and Therese Charles3
1Computer
Science Research Institute, School of Computing and Information Engineering,
University of Ulster, Coleraine, Northern Ireland BT52 1SA
2Centre for Health and Rehabilitation Technologies, School of Health Sciences,
University of Ulster, Jordanstown, Northern Ireland BT37 0QB
3SilverFish Studios, Coleraine, Northern Ireland BT52 2NR

2. Introduction
• Our interest is in low cost, technology based,
games enhanced physical rehabilitation
• Previous work has used webcams and Kinect
– Effective but has issues, e.g. lag and resolution
• Leap Camera released
– Low lag, high resolution tracking of fingers
– We investigated beta camera and sdk
– Developed several VR therapies for fingers.
• In this paper we present results from trials with
professional physiotherapists

3. Previous Research
• Video of previous webcam and augmented
reality games from Ulster ->

4. Leap Motion Camera/Controller

5. Leap Motion camera/controller
• Low cost (approx. £70 in the UK)
• Small (0.5” x 1.2” x 3” with a weight of 0.1
pounds)
• Built-in infrared LEDs to detect objects within
a dome of approximately 8 cubic feet above it
• Minimal latency and high spatial precision
(0.01mm)

6. Leap Demo
• On Screen (Visualizer + other examples)

7. Method
• Construct virtual simulations of 3 common rehab tasks
for hand and fingers
– Leap Motion and a natural interface controller
– Use a 3D game engine for construction
– Collaborate with commercial developers (SilverFish
Studios) and academics from the Centre for Health and
Rehabilitation Technologies (CHaRT)
• Trial simulated tasks with clinicians
– Give us understanding of the potential and limitations of
the Leap
– Obtain feedback from professionals at Regional Acquired
Brain Injury Unit (RABIU) at Musgrave Hospital in Belfast
before considering patient trials

8. Task 1 – Cotton Balls
Activity Description
Progression Parameters
Feedback Parameters
Cotton balls and a container are placed on a
table. The user is asked to pick the cotton balls
up off of the table and place them in the
container. The use is encouraged to use a pincer
grasp.
• Size of Container
• Distance of container from user
• Height of container
• Number of cotton balls
• Time required to place a set number of
cotton balls in the container
• Number of repetitions

9. Task 2 – Stacking Blocks
Activity Description
The user is given rectangular blocks
(wooden or plastic), and is asked to
build a tower by stacking blocks
vertically on the table.
Progression
• Distance of blocks from the user.
Parameters
• Number of blocks
• Size of blocks
Feedback Parameters • Number of repetitions in a set time
period
• Time: Time required to complete the
task

10. Task 3 - Nine Hole Peg Test (NHPT)
• The standardized equipment for the test typically
consists of:
– A board, in wood or plastic, with 9 holes (10 mm
diameter, 15 mm depth), placed apart by 32 mm
(Mathiowetz et al. 1985) or 50 mm (Heller et al. 1987)
– A container for the pegs. Initially the container was a
square box (100 x 100 x 10 mm) apart from the board.
The most current container is a shallow round dish at
the end of the board (Grice et al., 2003)
– 9 pegs (7 mm diameter, 32 mm length) (Mathiowetz
et al. 1985)
– Stopwatch

11. Virtual Tasks
• Video or live demo of software
Screenshots of three simulated rehab tasks. From left to right: Cotton Balls, Nine Hole Peg Test, and
Stacking Blocks.

12. Trial Participants
Participant
Q1. Occupation
1
Physiotherapist
2
3
4
5
6
7
8
Q2. Years of
Experience
10+
Q3. Use Games
for Rehab
No
Q4. Play Games
Occupational
Therapist
Student
10+
Yes
Occasionally
0
No
Never
Occupational
Therapist
Occupational
Therapist
Occupational
Therapist
Occupational
Therapist
Occupational
Therapist
1-2
Yes
Occasionally
3-5
Yes
Occasionally
3-5
Yes
Never
5-10
Yes
Occasionally
5-10
Yes
Never
Occasionally

13. Post Trial Questions
Background information
1. What is your occupation?
Physiotherapist / Occupational therapist / Other
2. How many years clinical experience do you have?
3. Do you use computer games for rehabilitation? YES/NO
System specific questions
(Answers on a Likert Scale 1 – 7)
4. Do you play games?
5. I feel that with practise I would become proficient in using the control interface
6. I feel that with practise I would become proficient in using the control interface
7. The tasks presented on the screen are easy to understand
8. The content on the screen is appropriate for the patient population
9. The prototypes provide a good illustration of all the functionalities I would require it to
have e.g. type of tasks, movements emphasised
Patient population related questions
(Answers on a Likert Scale 1 – 7)
10. I feel it would be easy to use this system in my clinical environment
11. I can see the benefit of this system for my general patient population
12. I can see the benefit of this system for the older patient population
13. I can see the benefit of this system for the younger patient population
14. I feel patients would be motivated to use this system
15. I feel my patients would benefit from this type of system in their home environment
16. I feel the system needs to be adapted to suit my patient population
Please state how it would need adapted (follow up comment)

14. Full Results

15. Mean Responses per Question

16. Variation per Participant
Participant
Q1. Occupation
Q2. Years of
Experience
Q3. Use
Games for
Rehab
Q4. Play Games
Q5 – Q16
Mean
Response
Interquartile
Range
1
Physiotherapist
10+
No
Occasionally
1.33
0.75
2
Occupational
Therapist
10+
Yes
Occasionally
2.25
0.75
3
Student
0
No
Never
2.67
2.5
4
Occupational
Therapist
1-2
Yes
Occasionally
4.00
2
5
Occupational
Therapist
3-5
Yes
Occasionally
1.75
1.75
6
Occupational
Therapist
3-5
Yes
Never
4.17
1
7
Occupational
Therapist
5-10
Yes
Occasionally
1.00
0
8
Occupational
Therapist
5-10
Yes
Never
3.58
3
6 Yes
2 No
5 Occasionally
3 Never
2.59 (Mean)
1.47 (Mean)
Summary

17. Summary of Results
• Mean response to all questions = 2.59 (scale 1-7)
– Six of respondents had a mean response to the
questions of less than 3
• 4 clinicians over 5 years experience provided a
(mean = 2.04)
– 4 other responded with a (mean = 3.14)
•
•
•
•
Tasks easy to understand (mean = 1.88)
Good illustration of rehab tasks (mean = 2.38)
Suitable for young patients (mean = 1.6)
Suitable for home environment (mean = 2.5)

18. Discussion
• Ease of use of system
– We spent most time on this – it is tricky!
– Some clinicians did the interface tricky – especially at the start
• Issues: its an unfamiliar UI, hand position must remain parallel to table
surface (+/- 10 degrees or so)
• Lessons: user needs time to attune, lots of positions cues required, VR
headset like Oculus Rift may help
– Several clinicians felt that older people could have problems
learning to use the system
• Clinicians felt that the system has potential but the tasks
should be converted into games and could be more tailored
to their treatements
• Clinicians were excited about the use of the system in the
home – especially since it is so cheap and easy to set up.

19. Participant Comments
• “Develop programmes for different
orientation, i.e. vertical and horizontal”
(Participant 1)
• “More functional activities; i.e. lift a cup,
bringing to a mouth. Puzzles - to incorporate
cognitive skills” (Participant 4)
• “May be set games for different age groups”
(Participant 7)

20. Conclusion
• The results were very encouraging
• We have learned a lot about designed virtual
rehab. software that use the Leap controller
• Next phase
– Move software to Unity3D
– Design games for the Leap in collaboration with
clinicians
– Trial games with patients