1. Game Design to Enhance Cognitive Adaptability
ADL Webinar, 12/17/2013
Shane Gallagher, Ph.D., Serco in support of ADL
Shenan Prestwich, Katmai in support of ADL

2. What does your IQ mean?
116+
17 percent of the world population;
superior I.Q.; appropriate average
for individuals in professional
occupations.
121+
10 percent; potentially gifted;
average for college graduates
132+
2 percent; borderline genius;
average I.Q. of most Ph.D.
recipients
143+
1 percent; genius level; about
average for Ph.D.'s in physics
158+
1 in 10,000; Nobel Prize winners
164+
1 in 30,000; Wolfgang Amadeus
Mozart and the chess champion
Bobby Fischer.
20-50% of variation due to non-genetic
factors
Genetic factors not like those of eye color
but more like those for height and weight
www.nytimes.com/2012/04/22/magazine/can-you-make-yourselfsmarter.html?_r=1&adxnnl=1&page
2

3. Can cognitive capabilities be enhanced?
• What is brain
plasticity?
• Can the brain be
trained?
• Can intelligence be
increased?
• Fluid intelligence vs.
crystalized intelligence
3

4. Cognitive capabilities can be enhanced
According to research by:
• Klingbert (2005)
• Jaeggi & Buschkuel (2008)
• Green & Bavelier (2003)
• Maguire, et al (2000)
• Dranganski, et al (2004, 2006)
• Willis, et al (2006)
• Holmes (2009)
• And many others….
4

5. Can cognitive capabilities be enhanced…
through game design?
If so, how?
That is our question….
5

6. Background
Adaptability and Cognitive Adaptability
6

7. What Is Adaptability?
• Adaptability: important metacompetency identified
by U.S. DoD, DoL, DoE.
• Ability to use existing knowledge to create innovative
problem solutions
• Repeatedly trying new/different strategies to solve
problems while reflecting on actions and
incorporating feedback
7

8. What Is Adaptability?
• Present on a scale of macro to
micro
•
Macro: adaptive stance, operational
adaptability
•
Mid: individual adaptive behavior
•
Micro: micromomentary cognitive
processes, i.e., cognitive adaptability
8

9. Lens of Analysis
• Cognitive Adaptability
•
Micro (cognitive) Level
•
Components
•
Cognitive Openness/Creativity
•
Focused Attention
•
Cognitive
Flexibility/Metacognition
•
Critical Thinking & ProblemSolving
9

10. Cognitive Adaptability
10

11. How This Translates to Game
Design
11

12. Mechanics
Dynamics
Aesthetics
“…where mechanics are the components of
a game at the level of data representation
and algorithms, dynamics are how the game
components interact with the player and viceversa, and aesthetics comprise the emotional
response evoked by the mechanics and
dynamics…” (Hunicke, LeBlanc, & Zubek,
2002)
12

13. A Variation on MDA: FFMDA
13

14. FFMDA
14

15. FFMDA
15

16. Fostering Cognitive Adaptability
• Intersection of games and CA = research void
• Had to draw from other areas of research and posit
how these findings can be translated into game
design
Clinical Psychology
•
•
Cognitive Remediation Therapy (CRT)
Learning Sciences
•
•
CANTAB/Cambridge Cognition
•
Feature Overlap Theory
16

17. Lens of Analysis
5 Features for CA
• Unstated/Implicit Rules
• Unstated/Implicit Shifting of
Rule Sets
• Dynamic Shifting
Environments
• Open-Ended Gameplay
• Implicit Reinforcement for
Individual Actions/ Choices
to Achieve Final Goal
17

18. The Study
The Study
18

19. Goal
• Examine whether playing a commercial off-the-shelf
game with the five identified features will increase
cognitive adaptability
19

20. Design
Study site and population chosen
•
•
•
Sheppard AFB, 18-24 year olds enlisted and lieutenants (N=39)
Randomly assigned to study and groups
Design
•
•
•
Experimental – pre/post tests and
Correlational – history plus metacognitive awareness inventory
Pre/post measures
•
•
•
•
CANTAB customized battery
Metacognitive Awareness Inventory
Post measures
•
Game History Questionnaire
20

21. CANTAB
• Cognitive testing software
• Customized battery
•
Attention Switching
•
Spatial Working Memory
•
Spatial Span
•
Rapid Visual Information Processing
•
Executive Planning
•
Reaction Time (N/A)
21

22. Design
Intervention Game:
Portal II
•
Play for 6 hours over
2 days (12 hours
total) with breaks
every 1.5 hours
Control: Puzzle/Logic games commonly found
with Windows 7 (lacking the five features)
Solitaire, Minesweeper, Hearts, Bejeweled/Tetris or similar
game
Play each for 1.5 hours a day over 2 days (12 hours total)
22

23. Results
23

24. Portal 2 Play
• Cognitive focused attention (specifically signal detection) was
enhanced in Portal 2 players versus MS Games players.
• Potential cause is the amount of concentration needed to infer
implicit rules, their changes, and environmental changes.
24

25. Gaming Experience
• Previous experience playing Portal 2 within past six months
correlated with greater spatial working memory capabilities
• Historic gameplay of 19+ hours/week correlated with greater
spatial working memory, spatial sequencing, and cognitive
planning capabilities
• Design characteristics presents in MMOGs, card games, puzzles,
racing, strategy, and sports games may lead to these increases
and should be explored.
25

26. Evaluating Games for Cognitive
Potential
26

27. Cognitive Task Analysis (CTA)
• Technique to capture and model behavioral and cognitive
processes/activities for accomplishing a task at expert-level
• decision-making processes
• recognizing and responding to critical cues/environmental
conditions
• utilizing tools
• performing sub-tasks
• analyzing and altering one’s own performance
• Commonly used by industry and research professionals
• Human computer interaction, instructional system design
• Developing :
•
•
•
Intelligent/Cognitive Tutors
Decision Support Systems
Roots in cognitive science
27

28. Applying CTA to Video Games
• Need to understand empirically:
•
Design of game
•
How game design interacts with player cognition
• Using CTA to map and model cognition and
interaction within video game play is novel to the
literature.
• Researching methods for applying CTA to video game
analysis
28

29. Protocol: CTA & Video Games
Part I: Preliminary Analysis/Lexicon Development
•
Establish vocabulary of game (names for features, tools,
obstacles, etc.)
•
Game structure analysis: break game down into
measurable units based on narrative structure
29

30. 30

31. Protocol: CTA & Video Games
Part 2: Focused Knowledge Elicitation
Level-by-level:
•
•
•
Please play through this level, completing the necessary steps to achieve the final
objective. As you do, try to speak aloud as much as possible, narrating both your
thought process (including what decisions you are making and what options you are
considering) and the sequences of actions you take as you take them. As you play,
please survey the areas you are in and describe your expectations in terms of how you
will have to interact with the room in order to achieve your objective.
Gameplay recorded on FRAPS with audio plus
simultaneous .wav file recordings.
31

32. Protocol: CTA and Video Games
Part 2: Focused Knowledge Elicitation
Subject and researcher review FRAPS gameplay for the
following information about each level:
•
•
Affordances
•
Requisite Knowledge
•
Micro-Puzzles Present
•
Mechanical Steps Required
•
Cognitive Steps Required
•
Audio and Visual Cues
32

33. Example FRAPS: Ch. 1, Level 3
33

34. 34

35. Protocol: CTA and Video Games
Part 2: Focused Knowledge Elicitation
•
Reviewing FRAPS gameplay a second time, identify
presence or absence of design characteristics in question
35

36. 36

37. Progress and Results
• Protocol developed for CTA on video gameplay that
encompasses cognitive, mechanical, and design
elements
• Analysis shows modulating but persistent presence
of all five cognitive adaptability design characteristics
throughout Portal 2
• Indication of varying information processing/filtering
and executive function requirements throughout
game
37

38. Future Direction
• Developing a cognitive profile of all analyzed puzzles
and micro puzzles in Portal 2
• Aligning the puzzles to specific CANTAB battery
components
• Designing custom Portal 2 levels with Level Builder
with puzzles aligning to specific cognitive properties
• Fine tuning levels for specific cognitive enhancement
properties
• Using designed levels as an intervention for CA
increase
38

39. Resources
Supporting Cognitive Adaptability Through Game Design
http://www.adlnet.gov/resources/supporting-cognitive-adaptability-through-gamedesign?type=research_paper
Can Game Design Be Leveraged to Enhance Cognitive Adaptability?
http://www.adlnet.gov/wp-content/uploads/2013/02/CA-Games-Design-EJeL-GAL-FNL020313.pdf
Can Playing Video Games Improve Cognition and Adaptability?
http://www.adlnet.gov/from-adl-team-member-shane-gallagher-can-playing-video-gamesimprove-cognition-and-adaptability
The Cognition of Gameplay
http://www.adlnet.gov/from-adl-team-member-shenan-prestwich-the-cognition-of-gameplay
Cognitive Task Analysis: Analyzing the Cognition of Gameplay and Game Design
https://app.box.com/s/85tvzlmwv96v74yg1xnu
Transforming Education Through Neuroscience, Cognition, and Game Design
http://www.adlnet.gov/wp-content/uploads/2013/09/Transforming_Education_ITX3.pdf
Example Video
Portal 2 Cognitive Task Analysis: Ch. 1, Level 3
39

40. Contact Us
Shane Gallagher
shane.gallagher.ctr@adlnet.gov
Shenan Prestwich
shenan.prestwich.ctr@adlnet.gov
40