O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
b
b
© Know-Center GmbH, www.know-center.at
In-App Reflection Guidance for
Workplace Learning
Angela Fessl, Gudrun Wesiak, ...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
Introduction
• In-App reflection guid...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
Introduction
• Conscious re-evaluatio...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
Background – Reflection Guidance
• RL...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
Reflection Guidance Concept
• Reflect...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
Reflection Guidance Concept
• Reports...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
Reflection Guidance Concept
• Context...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
Methodology
User studies in real work...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
Methodology
• Log-Data  objective us...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
Methodology
• Analysis of written con...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
User Studies
• MoodMap App (Call Cent...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
User Study: MoodMap App at Call Cente...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
User Study: MoodMap App at Call Cente...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
User Study: MoodMap App at Call Cente...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
User Study: Medical Quiz at Hospital
...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
User Study: Medical Quiz at Hospital
...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
User Study: Medical Quiz at Hospital
...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
User Study: KnowSelf at IT Company
• ...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
User Study: KnowSelf at IT Company
19...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
User Study: KnowSelf at IT Company
• ...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
Results
• MoodMap App: 33 participant...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
Results: Usage and Usefulness of Refl...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
Results: Usage and Usefulness of RGC
...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
Results: Usage and Usefulness of RGC
...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
Results: Content Analysis of inserted...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
Conclusion
1. How were the implemente...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
Conclusion
• Correct timing of reflec...
© Know-Center GmbH
gefördert durch das Programm COMET (Competence Centers for Excellent Technologies), wir danken unseren ...
© Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics
References
• Prilla, M., Renner, B.: ...
Próximos SlideShares
Carregando em…5
×

Ec tel2015: In-app Reflection Guidance for Workplace Learning

In-app reflection guidance for workplace learning means motivating and guiding users to reflect on their working and learning, based on users’ activities captured by the app. In this paper, we present a generic concept for such in-app reflection guidance for workplace learning, its implementation in three different applications, and its evaluation in three different settings (one setting per app). From this experience, we draw the following lessons learned: First, the implemented in-app reflection guidance components are perceived as useful tools for reflective learning and their usefulness increases with higher usage rates.
Second, smart technological support is sufficient to trigger reflection, however with different implemented components also reflective learning takes place on different stages. A sophisticated, unobtrusive integration in the working environment is not trivial at all. Automatically created prompts need a sensible timing in order to be perceived as useful and must not disrupt the current working processes.

  • Seja o primeiro a comentar

  • Seja a primeira pessoa a gostar disto

Ec tel2015: In-app Reflection Guidance for Workplace Learning

  1. 1. b b © Know-Center GmbH, www.know-center.at In-App Reflection Guidance for Workplace Learning Angela Fessl, Gudrun Wesiak, Verónica Rivera-Pelayo, Sandra Feyertag, Viktoria Pammer EC-TEL 2015, Toledo, 17.09.2015
  2. 2. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics Introduction • In-App reflection guidance concept to trigger reflective learning • … at the workplace 2 Purpose of this work
  3. 3. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics Introduction • Conscious re-evaluation of past situations/experiences with the goal to learn from them & use the gained outcomes to guide future behaviour • Workplace learning: core process to get new insights, derive better practices -> improve own work • Cognitive process based on intrinsic & extrinsic motivation of an individual 3 Reflective Learning
  4. 4. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics Background – Reflection Guidance • RL Tools: Prompts, Reflection Amplifiers, Diaries, Journals, e- Portfolios • Prompting approaches successful in formal learning environments • Journals/Diaries in educational settings e.g. nursing, athletics • Little research yet in work-related settings • Challenges: • Work tasks not always known beforehand/vaguely known • Right time to display prompts? • Working environment is often stressful & work-intensive 4
  5. 5. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics Reflection Guidance Concept • Reflection interventions • Prompts to motivate users to utilise an application in general • At beginning/end of day, at the start of an event or when application was not used for a longer time • Reflection amplifiers • Well-considered prompts  initiate reflection & support planning and reflection • Make users aware of unusual behaviour/working patterns or other significant situations • Motivate users to insert their thoughts into corresponding application 5 Reflection-in-Action Components (Verpoorten et al., 2010)
  6. 6. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics Reflection Guidance Concept • Reports • Automatic summary of experiences/events captured during work • Hourly, daily, weekly • Compare them with own or other‘s working behaviour  detect patterns & insights  learning outcomes • Reflection Diary • Store insights, thoughts, artefacts, learning outcomes in structured way • Re-evaluation of entries  trigger reflective learning & lead to insights guiding future behaviour 6 Reflection-on-Action Components
  7. 7. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics Reflection Guidance Concept • Contextualisation • Manually or automatically tracked by application • Automatic tracking: extensive amounts of data • Reflective learning support: • User has to think about context information when entering it • Context information when revisiting captured data helps remembering the respective working situation better 7 Interwined Component
  8. 8. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics Methodology User studies in real work settings (3 companies, 3 Apps with reflection guidance components) 1. How were the implemented reflection guidance components perceived with regard to their usefulness at work? 2. To what extent (at which stage) did reflective learning occur? 8 Research Questions
  9. 9. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics Methodology • Log-Data  objective usage rates • Pre-Questionnaire: demographic data • Post-Questionnaire • Support of apps with respect to RL • Perceived usefulness of reflection guidance components • Interviews & workshop: deeper information 9 Analysis Data
  10. 10. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics Methodology • Analysis of written content: qualitative analysis schema (independent analysis & categorisation by 3 researchers) • 1. stage: descriptions of experiences & emotions • 2. stage: interpretation & justification of actions and working on solutions • 3. stage: insights, learning & conclusions from reflection 10 Evaluations (Prilla et al., 2014)
  11. 11. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics User Studies • MoodMap App (Call Center) • Insights on how mood influences work performance & collaboration within a team • Medical Quiz (Stroke Unit) • Refresh existing knowledge & gain new knowledge • Connect theoretical knowledge with working practices & reflection • KnowSelf (IT Company) • Support of individual reflective learning regarding time management & self-organisation of knowledge workers 11 Overview and Learning Goals
  12. 12. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics User Study: MoodMap App at Call Center • Maps mood on a coloured map along 2 dimensions: • Valence (feeling good – feeling bad) • Arousal (high energy – low energy) • Reflection Guidance: • Contextualisation: For each mood choose pre-defined context & add note (supported by reflection amplifier question, e.g. What currently influences your mood?) • Reports (mood development over day) & reflection diary 12 MoodMap App
  13. 13. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics User Study: MoodMap App at Call Center 13 MoodMap App
  14. 14. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics User Study: MoodMap App at Call Center • Call Center: • 4 teams of 2 call centers of a large British telecommunication company • Each team: • Call takers (take calls & solve issues directly with customers) • Coaches (support & coach call takers for their work) • Managers (supervice training sessions, review call takers‘ performance & ensure their performance against targets) • 4 weeks • All participants were asked to integrate MoodMap App in their daily work routines 14 MoodMap App – Setting
  15. 15. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics User Study: Medical Quiz at Hospital • Quiz especially developed for nurses at a Stroke Unit in German hospitals • 4 different quiz types • Randomised content-based questions • Reflection guidance: reflection amplifiers • At the beginning: • address current knowledge status and play frequency • In-between: • put user‘s focus on content-based questions • At the end: • ask for gained insights or new knowledge w.r.t. the played quiz 15 Medical Quiz
  16. 16. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics User Study: Medical Quiz at Hospital 16 Medical Quiz
  17. 17. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics User Study: Medical Quiz at Hospital • Qualification program for becoming nurses working for a Stroke Unit • 5 months • Education: 1 week per month • Nurses were asked to play quiz consequently • Strengthen newly gained knowledge • Reflect about content & draw connections between new knowledge and daily work practices 17 Medical Quiz – Setting & Evaluation Procedure
  18. 18. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics User Study: KnowSelf at IT Company • Automatic tracking of resources & apps used on PC • Manual project & task recording • Reflection Guidance: • Reflection diary for notes, thoughts & insights • Time-triggered prompts • Remind user of reflecting by offering feedback on different aspects of user‘s behaviour • Event-triggered prompts • After detection of a significant change (e.g. unusually high number of switches) 18 KnowSelf
  19. 19. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics User Study: KnowSelf at IT Company 19 KnowSelf
  20. 20. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics User Study: KnowSelf at IT Company • Consulting Company • Consults, sells & personalises CRM (Customer Relationship Management) Software • Reflection regarding interactions with customers • Installation on business PCs • Reflect on collected data on daily basis over 6 weeks • Weekly reminder • Reflect on captured data • Enter observations • Fill in short questionnaire 20 KnowSelf - Setting
  21. 21. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics Results • MoodMap App: 33 participants • Interaction: M=42.67 (SD=72.2) from max. 338 to min 1 • Reflective Learning: M=3.25 (SD = .92) • Medical Quiz: 18 participants • Questions answered: M = 461.9 (SD=341), from max 1358 to min. 25 • Reflective Learning: M = 3.51 (SD=.42) • KnowSelf: 10 participants • Usages: M=38.5 min (SD 23.8), from max. 53.72min to min. 7.76min • Reflective Learning: M = 3.28 (SD =.66) • Sufficient app usage to evaluate reflection guidance components • Components perceived as useful and reflective learning has taken place • Positive correlations between extent of app usage & users‘ perceived support for reflective learning (significant for Medical Quiz & KnowSelf) 21 Did participants use apps, and did they actually reflect?
  22. 22. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics Results: Usage and Usefulness of Reflection Guidance • Context distribution: • After a call: 31% • After a break: 6% • After a coaching session: 2% • Other: 61% • How helpful was the contextualisation for • Associating inserted moods to certain situations: M=3.39 (SD=.86) • Recalling past experiences: M=3.21 (SD=.74) • Helping to better reflect about past situations: M=3.24 (SD=.94) • Context should be adaptable to individual preferences 22 Contextualisation – MoodMap App
  23. 23. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics Results: Usage and Usefulness of RGC • Time-triggered prompts • Reminder to use app: M=3.22 (SD=1.09) • Reminder of project recording: M=2.75 (SD=1.28) 23 Reflection Interventions - KnowSelf
  24. 24. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics Results: Usage and Usefulness of RGC • MoodMap App: Add note to moods • 475 notes inserted • Medical Quiz: reflective questions • 1205 posed, 603 (52%) meaningful answered • 59.3% at beginning, 51.5% at end, 42.4% in-between • Willingness to reflect is higher at beginning and end of the quiz than inbetween (disruptive) • KnowSelf: event-triggered prompts • Notification about most used resources: M=3.30 (SD=1.25) • Notification about unusual amount of idle time: M=2.57 (SD=0.79) • Notification about specific amount of switches: M=2.30 (SD=1.34) • Asking the right questions at the right time can trigger reflective learning 24 Reflection Amplifiers: MoodMap App, KnowSelf, Medical Quiz
  25. 25. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics Results: Content Analysis of inserted Notes • MMA: 457 notes to 548 inserted moods: 283 individual reflective items • 95% 1.stage (descriptions of experiences & emotions) • 5% 2.stage (interpretations/justifications of actions & working on solutions) • KnowSelf: 103 statements  33 non-reflective, 11 no inter-coder agreement, 59 reflective items • 22% 1.stage (descriptions of experiences & emotions) • 47% 2.stage (interpretations/justifications of actions & working on solutions) • 27% 3.stage (insights, learning outcomes, conclusions from reflection) • Medical Quiz: Answers to open questions were too short for analysis (often only a single word) 25 Stages of reflection
  26. 26. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics Conclusion 1. How were the implemented reflection guidance components perceived with regard to their usefulness at work? • Components perceived as useful • In-app reflection guidance facilitates technology-supported reflective learning at work 2. To what extent (at which stage) did reflective learning occur? • Achieved reflection stages differed between the apps and setting • MoodMap App: • 1. stage (experience) & 2. stage (interpretation) • KnowSelf • 1. stage (experience) & 2. stage (interpretation) & 3. stage (insights) 26 Research Questions
  27. 27. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics Conclusion • Correct timing of reflection interventions & amplifiers is crucial -> avoid disturbing interruptions • Interruptibility: ongoing research challenge • Give users control over notification 27 Lessons Learned
  28. 28. © Know-Center GmbH gefördert durch das Programm COMET (Competence Centers for Excellent Technologies), wir danken unseren Fördergebern: bQUESTIONS Angela Fessl afessl@know-center.at Know-Center GmbH Inffeldgasse 13 8010 Graz 28
  29. 29. © Know-Center GmbH • Research Center for Data-Driven Business and Big Data Analytics References • Prilla, M., Renner, B.: Supporting collaborative reflection at work: a comparative case analysis. In: Proceedings of ACM Conference on Group Work (GROUP 2014). ACM (2014) • Schön, D.A.: The Reflective Practitioner: How Professionals Think In Action, 1st edn. Basic Books, New York (1984) • Verpoorten, D., Westera, W., Specht, M.: Reflection amplifiers in online courses: a classification framework. J. Interact. Learn. Res. 21(4), 654–666 (2010). 29

×