We present GymSkill, a personal trainer for ubiquitous monitoring and assessment of physical activity using standard fitness equipment. The system records and analyzes exercises using the sensors of a personal smartphone attached to the gym equipment. Novel fine-grained activity recognition techniques based on pyramidal Principal Component Breakdown Analysis (PCBA) provide a quantitative analysis of the quality of human movements. In addition to overall quality judgments, GymSkill identifies interesting portions of the recorded sensor data and provides
suggestions for improving the individual performance, thereby extending existing work. The system was evaluated in a case study where 6 participants performed a variety of exercises on balance boards. GymSkill successfully assessed the quality of the exercises, in agreement with the
professional judgment provided by a physician. User feedback suggests that GymSkill has the potential to serve as an effective tool for motivating
and supporting lay people to overcome sedentary, unhealthy lifestyles. GymSkill is available in the Android Market as "VMI Fit"
Call Girls Bangalore Just Call 9907093804 Top Class Call Girl Service Available
GymSkill - A Personal Trainer for Physical Exercises
1. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
GymSkill:
A Personal Trainer for Physical Exercises
Andreas Möller, Luis Roalter, Nils Hammerla, Patrick Olivier,
Stefan Diewald, Johannes Scherr, Thomas Plötz
Matthias Kranz
Technische Universität München, Newcastle University,
Germany United Kingdom
March 22
PerCom 2012, Lugano, Switzerland
2. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
Outline
• Motivation
• Automatic Assessment of Physical Exercises
• Case Study
• Conclusion
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 2
3. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
Motivation
• Physical activity is required for healthy lifestyle
• Problem: people do not exercise enough
– Lack of knowledge of correct exercise execution for fast improvement
– High level of long-term motivation needed
• Advantages of a personal trainer:
– Continuous supervision and professional feedback
– Individualized advice and motivation
– Minimization of injury risk
• Disadvantages of a personal trainer:
– Expensive
– Privacy
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 3
4. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
Related Work
• Scientific focus on
– Activity recognition
– Wearable sensors
• Health and Fitness Systems
– Heart rate monitor, foot pod, GPS watch, …
• No skill assessment
– Fixed activity tracking (Wii balance board, Kinect)
• Gaming aspect
• Dedicated hardware
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 4
5. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
GymSkill
• Smartphone-based personal trainer
• Monitoring and assessment of physical exercises
– Based on phone sensor data
– No need for additional sensors
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 5
6. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
Outline
• Motivation
• Personal Health and Fitness Systems
• Automatic Assessment of Physical Exercises
• Case Study
• Conclusion
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 6
7. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
GymSkill: Automatic Assessment of Physical Exercises
• We look at rocker board exercises
• Example use case for equipment-based training
• Defined quality criteria:
tilt angles, speed, smoothness of movement, touching the ground…
• Phone is attached to the board
• During exercise: Basic situated feedback
• After exercise: Fine-grained analysis of exercise quality and reasons for
quality differences
move back and forth move left and right balance on the center
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 7
8. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
Skill Assessment Principle
Smartphone Server
“Cloud“
Sensor Data Recording
Log File
Sensor Data Processing PCBA Analysis
Simple Analysis
Real-Time Feedback Skill Assessment
HTTP
Skill Level (Score)
AJAX
Detailed Skill Report Output Rendering
User Feedback
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 8
9. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
Direct Feedback While Training
• Repetition count
• Visual feedback of board movement
• Warning when tilted too far
• Optional acoustic feedback
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 9
10. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
Post-Exercise Analysis
• Performed for each exercise run (e.g.: 10 repetitions of tilting back and forth)
• Global Analysis
– Smoothness and continuity of movement
– Global motion quality
– Usage of board‘s degrees of freedom
• Local Analysis
– Identify „interesting portions“ of sensor data
– What is „interesting“?
In recurrent data, this means unusual data compared to the rest
– E.g. participant hesitates or gets stuck
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 10
11. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
Global Analysis
• Estimation of motion axis (providing the dominant signal)
• Comparison of the empirical distribution
to ideal distribution function („gold standard“)
• Usage of normalized and un-normalized functions to determine
smoothness and utilization of board‘s degrees of freedom
• Transformation into a performance quality metric between 0 and 1
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 11
12. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
Local Analysis
• Assumption: sensor data of a movement shares (unknown) statistical
properties
• Unusual portions of a sequence violate this assumption and can be identified
• Extension to PCA: Principal Component Breakdown Analysis
• PCA model is learned from local neighborhood
(using sliding window technique)
• Frames are projected to lower-dimensional subspace using PCA
• Reconstruction errors used as a measure for motion quality
• Problem: ideal window size not known
• Solution: multi-scale comparison (iteratively growing window)
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 12
13. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
User Feedback
• Visual feedback
– PCA-based assessment diagram
(red and yellow parts contain irregularities)
• Textual feedback
– Based on global and local metrics Try
to
be
more
– Rule-based combination of aspects trigger continuous
in
textual cues your
motion!
• “Thumb” feedback
– Overall assessment at a glance
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 13
14. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
Outline
• Motivation
• Automatic Assessment of Physical Exercises
• Case Study
• Conclusion
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 14
15. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
Case Study
• Set of 20 exercises
developed by sports medicine specialist
• 6 participants, 5 days of training
(20 different exercises twice a day)
• 1200 exercise records
• Goal 1: Collection of training data
– Identification of criteria and assessment
by physician
• Goal 2: Qualitative evaluation of prototype
– Questionnaire study
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 15
16. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
Case Study: Trial Assessment Example 1
PCBA: Continuity
5 10 15 20
Time [s]
General motion Angle usage
0.2 0.25 Try to be more continuous in your motion!
observed You touched the ground 3 times.
ideal 0.2
0.15 Your movement is not ideal.
frequency
frequency
0.15 − Move back and forth in a continuous motion.
0.1
0.1 Try to move similarly to both sides of the board.
0.05 − You do not utilise the full range of angles!
0.05
− You lean towards the front!
0 0
−2 −1 0 1 2 −max 0 +max
displacement [std] displacement [°]
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 16
17. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
Case Study: Trial Assessment Example 2
PCBA: Continuity
5 10 15
Time [s]
General motion Angle usage
0.2 0.2 Your movement is continuous, nice!
observed You did not touch the ground!
0.15 ideal 0.15 Overall you perform the movement nicely!
frequency
frequency
Try to move similarly to both sides of the board.
0.1 0.1
− You do not utilise the full range of angles!
0.05 0.05 − You lean towards the front!
0 0
−2 −1 0 1 2 −max 0 +max
displacement [std] displacement [°]
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 17
18. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
Qualitative Feedback
Fully agree
Not agree at all
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 18
19. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
Qualitative Feedback
• Feature wishlist
Fully agree
Not agree at all
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 19
20. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
Outline
• Motivation
• Automatic Assessment of Physical Exercises
• Case Study
• Conclusion
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 20
21. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
Summary
• Mobile skill assessment of overall exercise quality
• Identification of typical exercising errors
• “Personal trainer”
• Participant feedback indicates potential for long-term exercising motivation
Further work:
• Long-term study on training progress
• Assessment and feedback entirely on mobile phone
• Generalization of assessment model
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 21
22. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
GymSkill in Google Play
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 22
23. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
Thank you for your attention!
Questions?
?
?
andreas.moeller@tum.de
www.vmi.ei.tum.de/team/andreas-moeller.html
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 23
24. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
Paper Reference
• Please find the associated paper at:
https://vmi.lmt.ei.tum.de/publications/2012/percom2012-preprint.pdf
• Please cite this work as follows:
• Andreas Möller, Luis Roalter, Stefan Diewald, Johannes Scherr, Matthias
Kranz, Nils Hammerla, Patrick Olivier, Thomas Plötz
GymSkill: A Personal Trainer for Physical Exercises
In: 2012 IEEE International Conference on Pervasive Computing and
Communications (PerCom2012), Lugano, Switzerland, March 2012, pp.
213-220
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 24
25. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
If you use BibTex, please use the following entry
to cite this work:
@INPROCEEDINGS{6199869,
author={M"{o}ller, Andreas and Roalter, Luis and Diewald, Stefan and Scherr,
Johannes and Kranz, Matthias and Hammerla, Nils and Olivier, Patrick and
Pl"{o}tz, Thomas},
booktitle={Pervasive Computing and Communications (PerCom), 2012 IEEE
International Conference on}, title={GymSkill: A personal trainer for
physical exercises},
year={2012},
month={march},
volume={},
number={},
pages={213 -220},
doi={10.1109/PerCom.2012.6199869},
ISSN={},}
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 25
26. Fachgebiet Verteilte Multimodale Informationsverarbeitung
Prof. Dr. Matthias Kranz Technische Universität München
Image Sources
• Slide 5
– gpsreview.net
– amazon.com
– spieleradar.de
– golem.de
04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 26