This document discusses how learning analytics can help take a more scientific approach to evaluating serious games. It presents three examples: 1) Using the xAPI standard to track all in-game interactions, 2) Applying game learning analytics to collect, analyze, and visualize learner data from a first aid game pilot study, and 3) Conducting a case study using analytics from a game for teaching subway navigation to people with Down syndrome. The document argues that learning analytics can help create better games and formally validate their impact on learning.
9548086042 for call girls in Indira Nagar with room service
Using Learning Analytics to support a more scientific approach to Serious Games: Three Examples
1. Using Learning Analytics to support
a more scientific approach to
Serious Games: Three Examples
Baltasar Fernandez-Manjon
balta@fdi.ucm.es , @BaltaFM
e-UCM Research Group , www.e-ucm.es
Jornadas eMadrid, 2017, 04/07/2017
Realising an Applied Gaming Eco-System
2. Can Learning analytics help?
- Do games actually works?
- Usually, no full formal evaluation has been carried out
- Limited number of users
- Formal evaluation could be as expensive as creating
the game (or even more expensive)
- Difficult to deploy games in the classroom
- Teachers have very little info about what is happening
when a game is being used
3. xAPI Serious Games application profile
New standard interactions model
developed and implemented in
Experience API (xAPI) by UCM with ADL
(Ángel Serrano et al, 2017).
The model allows tracking of all
in-game interactions as xAPI traces
(e.g. level started or completed,
interactions with NPC or game items,
options selected, score increased)
https://www.adlnet.gov/serious-games-cop
4. Analytics and Game Learning Analytics
Game Learning Analytics
(GLA) for Serious Games:
- collect, analyze and
visualize data from learners’
interactions
Can GLA be systematized?
Realising an Applied Gaming Eco-System
5. Systematization of Analytics Dashboards
As long as traces follow xAPI format, these analysis
do not require further configuration!
Also possible to configure game-dependent analysis and
visualizations for specific games and game characteristics.
6. First Aid - CPR validated game
• Collaboration with Centro de Tecnologias Educativas de Aragon, Spain
• Identify a cardiac arrest and teach how to do a cardiopulmonary
resuscitation to middle and high school students
• Validated game, in 2011, 4 schools with 340 students
Marchiori EJ, Ferrer G, Fernández-Manjón B, Povar Marco J, Suberviola González JF, Giménez Valverde A. Video-
game instruction in basic life support maneuvers. Emergencias. 2012;24:433-7.
Available at http://first-aid-game.e-ucm.es
7. BEACONING GLA Pilot: Experiment description
• 227 students
• 1, 2, 3 and 4 year of ESO and 1 year of
BACHILLERATO
• From 12 to 17 years old
• Only 4 morning sessions
• Game rebuilt with uAdventure
• Included analytics using RAGE tracker
based on xAPI specification
9. Learning compared to original experiment
• Original experiment
with the game
• Original experiment,
control group
• Current experiment
(from 8 to 9.8 out of 15)
Lower learning but still
significative!
Replicability of Results
10. Predicting post test score
1.With pre test information + game traces
Greater importance of:
- score in pre test
- game habits
- interactions with game elements
11. Predicting post test score
2.Only with game traces
Greater importance of:
- interactions with game elements
- scores in game levels
Long-term goal: predict score solely with in-game actions and,
therefore, avoid the pre test.
12. Case study: Downtown
• Serious Game designed and develop to
teach young people with Down
Syndrome to move around the city using
the subway
• Evaluated with 45 people with cognitive
dissabilities
• Full Analysis of xAPI GLA info under way
• Audience: People between 15 and 40 y/o with
Down syndrom
13. Case Study: Downtown
• From user requirements to a game
design and its observables
• Know more about how and what is
learn by people with Down Syndrome
13
18. The experiment: initial validation
With students from 3 institutes (Madrid, Zaragoza, Teruel)
223 pre-post and gameplays (121 males, 102 females)
19. First results
Age average
Pre-test value
average
Post-test
value average14,2
6,385,72
Once validated Ministry of Education is
interested in creating a NOOC for teachers
training where the game is used
20. Integrating xAPI LA in games authoring
Previous game engine eAdventure (in Java)
• Helps to create educational
point & click adventure games
Platform updated to uAdventure (in Unity)
Full integration of game learning analytics
into uAdventure authoring tool
No extra effort required to integrate default analytics into uAdventure
games!
21. Game Learning analytics can help us to:
create better games and to (formally)
validate games
• Moving from pre-post to Learning
Analytics based evaluation
• To use games as assessments
Conclusions