This document discusses using game learning analytics (GLA) to help teachers understand student learning from educational games. It describes challenges like collecting student data while protecting privacy. Teachers often find dashboards complex, so GLA systems aim to provide clear, easy to understand visualizations. The document shares examples of GLA dashboards created for specific games on topics like cyberbullying and teamwork. Lessons learned emphasize making dashboards simple with clear labels and alerts to help teachers intervene when needed. The conclusion states GLA can help validate games, monitor classes, and empower teachers, but systems must also consider teacher requirements to increase adoption of serious games in schools.

1. Making Understandable Game
Learning Analytics for Teachers
Antonio Calvo Morata, Cristina Alonso Fernández,
Manuel Freire Morán, Iván Martínez Ortiz,
Baltasar Fernández Manjón
balta@fdi.ucm.es @baltaFM
https://www.slideshare.net/BaltasarFernandezManjon

2. Serious Games for Education: Advantages
Ideal tool for teaching
● Break the 10 minute attention span
● Immersive
● Free and safe exploration
● Perception of progress
● Engaging and motivating nature
● Short feedback cycle
Successfully applied in several contexts: military, health, business, etc.
What about teachers when you deploy serious games?

3. Serious Games for Education: Handicaps
Still not widespread adoption of Serious Games in general education due to:
● production cost
● games adapted to the curriculum
● Teacher role: technical requirements, familiarity with technology
From the technical perspective there is a lack of:
● standards for development
● standards for validation
● standards for deployment

4. Game Learning Analytics
Learning Analytics (LA) aims to provide insight about learners’ actions in
educational contexts to improve their learning process and contexts.
When applied to games, Game Learning Analytics (GLA) gathers information
from in-game interactions.
Use of GLA to help teachers and provide insight into learners action:
● improve learning process
● improve learning context
Require implying teachers into the GLA process design

5. Game Learning Analytics: Pipeline

6. Game Learning Analytics: Data Collection Issues
Data collection
● issues to share the collected data
● privacy issues (now key element in Europe)
● data collected influence dashboard outcomes
Teacher involvement:
● difficult to obtain teachers requirements
● don’t expect to receive information in real time
● in general, prefer basic information but easy to understand and act on it

7. Game Learning Analytics: Dashboard Issues
Dashboard design
● design of possible dashboards is complex
● requires pedagogical knowledge and game-specific information
● teachers are (generally) no experts in dashboard design
Changing dashboard requirements
● teachers will often request additional visualizations after use the game
Beyond stand-alone games
● use games as parts of larger courses
● requirements of analyzed the game as a part of a whole

8. GLA: Our Experience in EU H2020 projects

9. Technical Issues: Data Collection
● Experience API for Serious Games Profile (xAPI-SG)
● Pseudoanonymous tokens for students
○ Token = 4 uppercase letters
○ Token shared between tests and user interactions data
○ (re-test) Teachers can keep correspondence between tokens and real
student
https://xapi.e-ucm.es/vocab/seriousgames

10. xAPI
Application profile for
Serious Games

11. Issues: Teacher Implication / Dashboard Design
● Default dashboard with different visualizations
○ No requires setup
○ Display basic data for any game with xAPI-SG data
● Alerts and Warnings
○ Notifications in real-time
○ Easy and adapted to the specific game
○ Allow teachers know the problems of each user
■ stuck player, serious mistakes…

12. Issue: Dashboard Design/Changing Dashboard Requirements
● Provide a wide range of default analysis and visualizations
● Allow create customized game-specific analysis and visualizations
● Allow re-evaluate old data for custom analysis and dashboards
○ Display existing information in new ways
● We use:
○ Kibana
○ Elastic Search
Issue: Beyond Stand-Alone Games
● Multilevel analysis

13. Example 1: Conectado
Conectado:
● Serious game to:
○ raise awareness against cyberbullying
○ increase empathy
● Graphic adventure
● Students in the victim role
Sessions
● Pre-post test
● Interactions data collected

15. Conectado: Dashboard
Dashboards to cover
● progress
● decisions taken
● specific metrics by class and by student
Average friendship risk
Number of players per game day
Number of player that have taken each possible action
Maximum friendship risk

16. Conectado: Experiments
Deployed in 8 institutes
● +800 students
● +120 educational sciences students
● +80 teachers
Dashboard used in sessions as a control tool
● Check that data was collected
● Check students progress
● Evaluate acceptance of dashboards
by the teacher

17. Lessons Learnt
Teachers usually consider dashboards complex to use and to act on it!
Significative names on visualizations and labels
Explain the information displayed
Alerts and warnings are very helpful
Compact visualizations (no scroll, if possible)

18. Example 2: RAGE Game
Work in a Game Studio and you need to
improve teamwork
Thomas-Kilmann Conflict Mode Instrument
measure an individual's response to conflict
situations
● Assertiveness
● Cooperativeness
Specific analysis and visualizations
● TKI categorization for each player
● Biases

20. TKI Dashboard
Specific request by creators of the game in collaboration with teachers
● Thomas-Kilmann classifications
● Games shipped (team productivity)
● Awards won (team quality)
● Office morale
● Overall Thomas-Kilmann classification
● Pie chart for each bias
● Ratio of responses where it was averted or exhibited

21. Lessons Learnt
Improvement of alerts and warnings view to better know when the teachers need
to intervene to be more effective

22. Example 3: Beaconing Dashboards
Multilevel analytics
● hierarchical model to aggregate
information from minigames to
higher-level nodes (e.g. quests,
missions) using rollup rules
New visualizations for geolocalized games
using an extension of the xAPI-SG Profile for
location-based interaction data

24. Conclusions
Teacher as a key to increasing adoption of SG by schools
Requirements of GLA should also into account teachers requirements
Game Learning Analytics (GLA) uses
● validate SG
● monitor the class
● empowering the teacher
Dashboards should be clear and easy to understand by teachers
● an iterative process to improve understandability
Next step: apply to gamification environments

25. Thanks
Baltasar Fernández
Manjón
balta@fdi.ucm.es
@baltaFM