This slides were presented at the invited speech at the World Conference on Educational Media and Technology (EdMedia) which was held in Vancouver, BC, Canada on June 28-30, 2016. Abstract: Wearable computing and augmented reality are disruptive technologies. They fundamentally change the way we educate and train people to a master level of performance. With advanced sensors we can capture experience as it emerges. For example, a trainee can receive live guidance in the form of semi-transparent 3D hands that appear at the right place spatially and operated by a remote expert using sensor data. Captured guidance provides reference to scale, allowing repeated access to the information asynchronously at the right time and in the right place where it is most urgently needed. Expert guidance can be captured with wearable sensors and later re-enacted by trainees with augmented reality creating a believable illusion of a master-apprentice knowledge sharing. The captured experience therefore represents a new type of educational media that has properties of carrying both explicit and tacit knowledge. This new media helps to convert experience to knowledge and enable learning by bringing closer the theoretical knowledge and immediate experience, which are traditionally separated. Tailored content of captured experience can be presented with augmented reality using intuitive and immersive user interfaces. This can have a positive impact on mental processing and memorization, not only adding scaffolds for high performance, but also acting as a safety net preventing potential problems sensed in the environment. Learning how to master a complex task usually involves reflecting on your own performance, looking back at your behavior and comparing it to that of others. The goal of this new training methodology is to enable the full cycle of immersive experience observing an expert, training with and without guidance, and observing own performance.

1. Wearable Experiences
for Knowledge Intensive Training
Mikhail Fominykh, PhD
Europlan UK Molde University College
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 1
June 28-30, 2016 • Vancouver, BC, Canada
Wearable Experience: New Educational Media for Knowledge Intensive Training
Invited speech

2. 29/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 2
June 28-30, 2016 • Vancouver, BC, Canada
#WEKIT
#edmedia
#edmediaconf

3. About me
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 3
Project manager in
workplace training with
Augmented Reality
and Wearables
Associate Professor
and researcher in
Emergency Management
training
Adjunct Professor and teacher
in Virtual Reality
Software developer in
psychological
treatment training
Software developer in
welfare personnel
training
Software developer in
cooperation training for
medical students

4. Background:
Experiential learning
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 4
Mikhail Fominykh, Peter Leong, and Brenda Cartwright (2016/2017) A Study on Experiential Learning in a 3D
Virtual Environment within a Professional Counseling Distance Course, in progress.

5. Background:
Virtual Reality Recording
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 5
Real time class
R0
R1
R1
R2
Recording
Replay
Recording
Mikhail Morozov, Alexey Gerasimov, Mikhail Fominykh, and Andrey Smorkalov (2013) Asynchronous Immersive
Classes in a 3D Virtual World: Extended Description of vAcademia, DOI: 10.1007/978-3-642-38803-3_5

6. Background:
Experiential learning
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 6

7. Background:
Experiential learning
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 7

8. Photo by Mikhail Fominykh
Wearable Computing
Augmented Reality
Experience and Knowledge
WEKIT methodology

9. Information
communication
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 9
Experience-
communication
Images: http://www.unmuseum.org/, http://en.wikipedia.org/, http://digitalxtrememedia.com/;
https://www.youtube.com/watch?v=7d59O6cfaM0, https://www.microsoft.com/en-us/research/project/holoportation/

10. https://www.youtube.com/watch?v=qAsu-RoVgWc
AR?

11. https://www.youtube.com/watch?v=7d59O6cfaM0
AR!

12. Object-
centric
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 12
Action-
centric

13. Performance Augmentation
with wearables
sensor-sensory loop:
super-real experience
010000100001000010001010001111110
The guest
enacts experiences
The stager
produces narratives
WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING29/06/2016 13

14. Photo by Ravensbourne
Wearable Computing
Wearable computing pursues an interface ideal of a continuously
worn, intelligent assistant that augments memory, intellect,
creativity, communication, and physical senses and abilities*.
* Starner (2011) The challenges of wearable computing: Part 1, IEEE Micro, Volume 21, Issue 4, DOI: 10.1109/40.946681

15. Why Wearable Computing?
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 15
Better access and management of information
Portability during operation
Enable hands free or hands-limited use
Can run continuously; and attempt to sense the user’s
current context (Rhodes, 2000)
Focuses the user’s attention and presents information in an
unobtrusive, context-dependent manner (Kortuem et al.)

16. Wearability
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 16
Head mounted
(headsets, glasses,
ear attachments)
Hand, wrist and arm
mounted (gloves,
watches, rings,
bands)Chest, neck back
mounted
(belts, bands,
wests, shirts)
Leg, feet mounted (bands,
shoes)

17. Photo by Mikhail Fominykh
Augmented Reality
Augmented Reality provides an enriched view onto the physical
world, adding layers with contextually useful information,
delivered visually or by stimulating other senses.

18. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 18

19. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 19
+80 /
+100%

20. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 20
https://www.youtube.com/watch?v=YJg02ivYzSs

21. Photo by Mikhail Fominykh
Reality is more rough

22. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 22
Working example
Kaj Helina, Timo Kuulaa, Jaakko Karjalainena, Iina Aaltonena, Fridolin Wildb, Juhani Viitaniemia, Antti
Väätänena (2015) Usability of the ARgh! Augmented Reality system for on-the-job learning

23. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 23
Kaj Helina, Timo Kuulaa, Jaakko Karjalainena, Iina Aaltonena, Fridolin Wildb, Juhani Viitaniemia, Antti
Väätänena (2015) Usability of the ARgh! Augmented Reality system for on-the-job learning

24. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 24
Working example

25. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 25
Working example

26. What is WEKIT?
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 26
Funding: EU Horizon 2020, ICT-20 2015: Technologies
for better human learning and teaching
Budget: EUR 2.753.143,75
WEKIT Community https://wekit-community.org/
WEKIT project website http://wekit.eu/

27. Experience and knowledge
Learning = converting
experience to knowledge
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 27
Mikhail Fominykh, Fridolin Wild, Carl Smith, Victor Alvarez and Mikhail Morozov: "An Overview of Capturing
Live Experience with Virtual and Augmented Reality”, DOI: 10.3233/978-1-61499-530-2-298.

28. Experience and knowledge
Learning =
= converting experience to knowledge
immediate
experience
(aka ‘practice’)
information for a
master level
(aka ‘theory’)
separated
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 28

29. Experience and knowledge
Learning =
= converting experience to knowledge
immediate
experience
(aka ‘practice’)
information for a
master level
(aka ‘theory’)
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 29

30. Experience and knowledge
Learning =
= converting experience to knowledge
immediate
experience
(aka ‘practice’)
information for a
master level
(aka ‘theory’)
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 30

31. Experience and knowledge
Learning =
= converting experience to knowledge
immediate
experience
(aka ‘practice’)
information for a
master level
(aka ‘theory’)
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 31

32. Experience and knowledge
Learning =
= converting experience to knowledge
immediate
experience
(aka ‘practice’)
information for a
master level
(aka ‘theory’)
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 32

33. Experience and knowledge
Learning =
= converting experience to knowledge
immediate
experience
(aka ‘practice’)
information for a
master level
(aka ‘theory’)
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 33

34. Experience and knowledge
Learning =
= converting experience to knowledge
immediate
experience
(aka ‘practice’)
information for a
master level
(aka ‘theory’)
Experiencedlearner
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 34

35. Methodology
What is Wearable Experience?
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 35
The Wearable Experience training methodology aims to provide an
innovative learning method that is based on the idea of capturing
the experience of an expert and enabling trainees to wear it while
re-enacting, thus giving the trainee access to the tacit knowledge of
the expert and enabling master-apprentice knowledge sharing.
Capture Re-enact
Evaluate

36. Methodology
What is Wearable Experience?
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 36
The Wearable Experience training methodology aims to provide an
innovative learning method that is based on the idea of capturing
the experience of an expert and enabling trainees to wear it while
re-enacting, thus giving the trainee access to the tacit knowledge of
the expert and master-apprentice knowledge sharing.
Giuseppe Scavo, Fridolin Wild and Peter Scott(2015) The GhostHands UX: telementoring with hands-on
augmented reality instruction, DOI: 10.3233/978-1-61499-530-2-236
Capture
example

37. Methodology
What is Wearable Experience?
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 37
The Wearable Experience training methodology aims to provide an
innovative learning method that is based on the idea of capturing
the experience of an expert and enabling trainees to wear it while
re-enacting, thus giving the trainee access to the tacit knowledge of
the expert and master-apprentice knowledge sharing.
Wild, Perey, Helin, Davies and Ryan (2014) TELLME’s Learning Experience Model in action on the shop
floor of a textile weaving mill
Re-enact
example

38. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 38
Jan Schneider, Dirk Börner, Peter van Rosmalen and Marcus Specht (2015) Presentation Trainer:
Polishing your communication skills, DOI: 10.1007/978-3-319-24258-3_52
Evaluate
example

39. Mikhail Fominykh, Fridolin Wild, Carl Smith, Victor Alvarez and Mikhail Morozov: "An Overview of Capturing
Live Experience with Virtual and Augmented Reality”, DOI: 10.3233/978-1-61499-530-2-298.
Capture | Re-Enact
WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING28/06/2016 39

40. Technology
What is Wearable Experience?
An experience that is facilitated by devices located on or in
the body that can include:
tracking user activity, biometric and environmental data
matching and mapping user performance
augmenting user’s activity with another user’s data
adapting user’s activity and behavior
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 40

41. # 41
Maintenance workers
in Aeronautics

42. # 42
Imaging and diagnostic workers
in Healthcare IT

43. # 43
Sub-system integration,
operation & maintenance
in Space
Photo by European Space Agency

44. WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING29/06/2016 44

45. WEKIT methodology
Phase A
Capturing

46. WEKIT methodology
Phase B
Re-enactment

47. WEKIT methodology
Phase C
Evaluation

48. Goals
Capture of Expertise
What are relevant aspects of expertise and how to capture them?
Enable enactment of expertise
How to actively support the transfer of expertise to a learner?
Design of learning task
How should learning activities be designed to support / complement
expertise transfer?
Requirement generation
How should technology be designed to foster all of the above?
48WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING28/06/2016

49. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 49
Domain limited (Fernand and Herbert, 1996)
Failure to recall surface features (Schmidt and Boshuizen, 1993)
Inaccurate prediction judgement and advice (Hinds, 1999)
Inflexibility and bias (Sternberg and Frensch, 1992)
Qualities of Experts
Superior Mental Knowledge (De Groot, 1978)
Perceptual abilities (Lesgold et al., 1988)
Qualitative analysis (Klein, 1998)
Better strategies (Lemaire & Siegler, 1995)
Cognitive effort (Patricia 2003)
Shortcomings of Experts

50. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 50
Top-down approach
Identify tangible and intangible aspects relevant in all tasks
Find abstract key factors and rating scheme
Assess tasks and mechanisms according to factors and
schemes
WEKIT methodology
Bottom-up approach
Start to analyse concrete tasks in industrial settings
Classify them according to common scheme identifying
aspects of expertise and learning
Try to find mechanism suitable to support expertise
transfer

51. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 51
Task Classifications
Transfer
mechanism
Highprecision
Highspeed
Decisiontaking
Collaborative
tasks
Highmemory
load
Perceptualtask
Assertivetask
Dualtask
Performance
Managing
Uncertainty
HighSpatial
Task
Slow motion
Zoom 3 13
Think aloud protocol 1 8
Object enrichment 2
Contextualization 3, 9 3
Self awareness of physical state 14
Directed focus 15
Case identification 16
Spatial Simulation 18
In Situ Real-time Feedback 4 17
Mobile control
Virtual Post its 5 5
Haptic hints
5,
12
Virtual/Tangible Manipulation 6
3,
10,
11
Remote symmetrical tele-
assistance
7 7
Literature
1Schraagen, 1993
2Tang et al., 2003
3Quarles et al., 2008
4BMW AR system, 2010
5Sabine et al., 2011
6Lahanas V et al., 2014
7Chinthammit et al., 2014
8Lundgrén-Laine et al, 2010
9Curtis et al., 1998
10Seichter, 2004
11Wang, Chen, Gong, & Hsieh, 2007
12Khademi et al., 2012
13Mulloni, Dünser &Schmalstieg 2010
14Larkin, 1983
15Jarodzka et al., 2013
16Mulzoff, 1990
17Kotranza, Lind, Pugh & Lok 2009
18Martin-Gutierrez et al., 2010

52. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 52
Tasks
Cognitive
Perceptual
Developmental
Human
Machine
Environment
Space
Time
Target
Domains Dimensions
Top-down approach

53. What is WEKIT?
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 53
TEL community – driven project

54. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 54

55. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 55

57. jTEL summer schooler
jTEL summer schooler
jTEL summer schooler
jTEL summer schooler
jTEL summer schooler
jTEL summer schooler
jTEL summer schooler

58. 28/06/2016 https://wekit-community.org/ideas/

59. 28/06/2016
http://bit.ly/wekit-facebook
http://bit.ly/wekit-twitter
http://bit.ly/wekit-linkedin
http://bit.ly/wekit-plus
http://bit.ly/wekit-youtube
WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 59

60. Disclaimer
This project has received funding from the Europea n Union’s
Horizon 2020 research and innovatio n programme under
grant agreeme nt No 687669. http: //we kit. e u/
Q & A
#WEKIT
Presented by Mikhail Fominykh
mikhail.fominykh@europlan-uk.eu
http://www.mikhailfominykh.com/
http://www.linkedin.com/in/fominykh/
https://www.facebook.com/mikhail.fominykh/
http://slideshare.net/mfominykh/
28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 60