Slides from a Games for Change 2017 Learning Summit presentation by Erin Hoffman-John on methods for creating effective and compelling video games for learning.

1. design lessons
from 3 years
at glasslab
erin hoffman-john
@gryphoness

2. L E T ’ S G E T T H I S
O U T O F T H E WAY
• what happened/is happening
to glasslab now?
• I don’t know*
• it’ll be okay**
• everything I’m about to say is
about making games for
learning (not ‘meaning’,
‘change’ etc)
* there are signs
** probably, I mean what or who is ever okay really***
*** I’m ok

3. A L I T T L E
H I S T O RY
• a 3 year mission funded by the
Bill & Melinda Gates
Foundation & Macarthur
Foundation
• partners including EA, ETS,
Pearson, Institute of Play, SRI
• to bring AAA development
quality to educational games &
apply big data analysis via ECD
to live streamed game data
• …just a little ambitious

4. T H I S TA L K I S R E T U R N O F T H E K I N G
I’m going to quickly summarize prior talks I’ve given about GlassLab because I figured
you’d all be tired of this stuff - apologies to those who have not seen any of them
Games for Change 2014 keynote:
GlassLab, Collaborating for Change
Serious Play 2016: Why We Can’t
Have Effective Games (Yet)
Games for Change 2017:
Design Lessons from 3 Years at GlassLab

5. M Y S I D E O F I T • it was hella fun
• learned a ton
1 9 9 8 2 0 0 3 2 0 0 5 2 0 0 6 2 0 0 8 2 0 1 0 2 0 1 2 2 0 1 3 2 0 1 4
(I was 17)
• changed my life
• now I owe you
• super hard
• amazing people
2 0 1 5

6. I U S U A L LY S AY
1. learning is an emotion, not just a
cognitive process (emotion is the
symptom of the process’s success)
2. games are about emotion
3. to be worthwhile a game must
be a valid assessment
4. to teach a game must be
thoroughly scaffolded
5. to teach widely a game must
have multiple representations
for more on this, look up my ux week talk or serious play slides

7. I A L S O TA L K
A B O U T
• the development of Mars
Generation One: Argubot
Academy, and how we
invented a new kind of
game design built on
measurable performance
indicators*
* also see the serious play talk for this stuff

8. N O W I W O U L D S AY
1. be intentional
2. be evidence-based
3. start with the boss level
4. find wizards and learn from them
5. involve learners in your process

9. L E T ’ S U N PA C K A L I T T L E
1. be intentional
• game developers are usually shockingly
unintentional - they’re intuitive, fluid, instinctive
• this totally will not work with learning*
• learning approaches are often TOO intentional -
too inflexible, too demanding
• you have to thread the needle: have a (precise,
concise) intent, then use game design
methodologies to hit it
* see my side talk “serious symbiosis” for the challenges of
getting game & learning designers to work together effectively

10. L E T ’ S U N PA C K A L I T T L E
2. be evidence-based*
• think about how you will test your efficacy from the
beginning. what will it teach, and how will you know? what
will the player be able to do when you’re done? how does
your solution compare to the best existing teaching
methods?
• ask an assessment expert what you’re not thinking of
• don’t make claims. have hypotheses. have partners
evaluate. the market will be averse to you doing this.
• the market is full of bullshit because we aren’t evidence-
based. it will poison this field if left unchecked.
• pretty != functional. twitterpated != educated.
* the methodology we developed was published as evidence-centered
game design (ECgD), a child of Bob Mislevy’s ECD

11. L E T ’ S U N PA C K A L I T T L E
3. start with the boss level
• if you’re doing it right you will spend 75% of your pre-
production defining your boss level
• if you are being evidence-based, this boss level will be
inescapable and complete: it will be a valid assessment
• if it ISN’T, you should not go forward
• when you have the boss level, you are only 25% of your
way toward having a (learning) game - because the boss
level doesn’t teach
• you might run out of money just on the boss level - and
that could be okay*
* arguably we should have shipped Mars Generation One with just the boss battles

12. L E T ’ S U N PA C K A L I T T L E
4. find wizards and learn from them
• get really real about admiring teachers*
• find the best teacher you can and observe them.
you will realize software can’t come close. but we
can simulate pieces of the experience.
• talk to average or even below average teachers
and listen to them. don’t persuade. listen. focus
on problems. remember that playtesters lie.
make it safe for them to tell you the truth.
• make something useful.
* no joke MJ and I admired teachers so hard we turned into them

13. L E T ’ S U N PA C K A L I T T L E
5. involve learners in your process
• I initially left this out because it seemed so obvious, but it’s
not - you’d be shocked -
• The presence of the learner changes the whole dynamic of
your development. Your ego will want to push off exposing
them to the process, but you need to resist this - the game is
never ready for the player, and always needs them
• In the end we described our games as co-designed with
learners, and this wasn’t (for me) PR - this was key to the
process. If you aren’t seeing them at least once a week,
you’re losing touch, you’re othering them.
• We didn’t do this enough, especially in the beginning - it
took us 2 years to hit the cadence.

14. A B R I E F
D I G R E S S I O N
• if I could give all of you just one
thing, it would be an hour with Bob
Mislevy, keeper of ECD
• this guy changed how we think
about games, learning, and
assessment
• if you ever get a chance to hang out
with him, do. he’s crazy smart and
an actual yoda

15. A N D A L S O
1. playtesters are full of lies
2. the game must have a core
3. know what you can afford
4. know who the player is
before you start
game design fundamentals you
can’t shortcut
like most game-crafting principles, these
seem simple and are not, not, not

16. L E T ’ S G E T M E TA
• you can’t bullet point concepts
• you have to take one concept and
go deep
• you (very likely) don’t have the
resources for several concepts
• learning is not a listicle
this talk is an example of what
not to do

17. you’ll note that I haven’t talked about money (yet)

18. I F Y O U WA N T T O M A K E M O N E Y
A T T H I S , M Y A D V I C E I S :

19. G I V E U P
I F Y O U WA N T T O M A K E M O N E Y
A T T H I S , M Y A D V I C E I S :

20. G I V E U P
I F Y O U WA N T T O M A K E M O N E Y
A T T H I S , M Y A D V I C E I S :
(just kidding)

21. G I V E U P
I F Y O U WA N T T O M A K E M O N E Y
A T T H I S , M Y A D V I C E I S :
(just kidding)
(but really you should seriously
consider giving up)

22. H E R E ’ S T H E
T H I N G
• GlassLab pivoted to building a
platform in year 2 because that was
the only hope for financial viability: a
path to schools (or parents)
• That platform didn’t make it
• The only companies who have made
this work have salesforces >100 bodies
• Someone will need to break this ice in
order to get games into classrooms at
sufficient scale
• And along the way, they’ll have to woo
and persuade and win over teachers
I scoured the
internet
trying to find
the name of
this artist and
couldn’t -
sorry, artist :(

23. W H I C H I S T O S AY
• the path to games in school is not going to be direct

24. W H I C H I S T O S AY
• the path to games in school is not going to be direct
…but that doesn’t mean it’s impossible

25. W H I C H I S T O S AY
• the path to games in school is not going to be direct
…but that doesn’t mean it’s impossible
• but you have to be very clear about what you are and
what your goals are
• hint: you are not a pegasus (sorry)

26. I N T E R M S O F M A R K E T, G L A S S L A B FA I L E D
B E C A U S E I T WA S T RY I N G T O B E A P E G A S U S *
• slow
• reliable
• thorough
• risk-averse
• efficient
Horses (aka learning
companies) are:
Birds (aka game
companies) are:
• fast
• unpredictable
• flighty
• risk-loving
• expensive
pegasus** isn't real (sorry)
* and in fairness, due to the requirements of our grant, we had to
** glasslab popup trivia: our first telemetry system was called “pegasus”

27. a commercial
game
T H E L E A R N I N G G A M E
C O N T I N U U M
a teaching
game
where are you?

28. a commercial
game
T H E L E A R N I N G G A M E
C O N T I N U U M
a teaching
game
ST MathDOOM SimCity Civ
where are you?
Slice Fractions

29. a commercial
game
T H E L E A R N I N G G A M E
C O N T I N U U M
a teaching
game
ST MathDOOM SimCity Civ
where are you?
Slice Fractions
“smart fun” zone

30. a commercial
game
T H E L E A R N I N G G A M E
C O N T I N U U M
a teaching
game
ST MathDOOM SimCity Civ
where are you?
Slice Fractions
“smart fun” zone
danja zone
(aka better be subsidzed)

31. a commercial
game
T H E L E A R N I N G G A M E
C O N T I N U U M
a teaching
game
ST MathDOOM SimCity Civ
where are you?
Slice Fractions
“smart fun” zone
danja zone
(aka better be subsidzed)

32. D O K U D O
• expressive social word game
preparing for soft launch (overseas
testing)
• building words out of graphemes -
not just letters (every word game
you’ve ever played is wrong for
effective literacy)
• social features: your most unique
word, the network’s best word of the
day
• edu version will focus on stealth
adult literacy

33. G L A S S L A B
M E T H O D 1 . 0
• find the leverage point
• find the leverage point inside
the leverage point
• find the single concept inside
the leverage point inside the
leverage point
• find the closest existing video
game mechanic to the
performance of that concept
• begin prototyping

34. D O K U D O M E T H O D
( G L A S S L A B 2 . 0 ? )
• start with a learning
principle
• spec as a “pure” edu
game
• begin development
• make all in-situ decisions
as commercial game
decisions

35. 1 . S TA R T W I T H A
L E A R N I N G
P R I N C I P L E
• our literacy SME told me something that
blew my mind: if you teach 36 letter-
vowel pairs (44 phonemes) instead of
just 26 letters, the pronunciation of
English stabilizes for 95% of words
• this is the secret to English literacy, and
it’s working for the schools that are
ahead, but it is not how English is taught
broadly still even in the US
• “we know how to teach English”, she
said - “and we’re not doing it”
• this was our leverage point
http://theconversation.com/the-way-we-teach-most-children-to-read-sets-
them-up-to-fail-36946

36. 2 . S P E C A S A
“ P U R E ” E D U
P R O J E C T
• this happened sort of by
accident: our SBIR
proposal (and the project’s
first, terrible name)
• we got rejected (maybe
fortuitous) because, get
this, it wasn’t believable
that people make money
off of mobile games

37. 2 . B E G I N
D E V E L O P M E N T
• get to prototyping
• (and the second, almost as
terrible, name
[“syllablocks”])
• despite being unfunded,
this game idea kept sticking
with us - and we couldn’t
believe there weren’t any
grapheme-based word
games on the market

38. 4 . W H I L E D E V E L O P I N G , M A K E
M O M E N T- T O - M O M E N T
D E C I S I O N S B A S E D O N
C O M M E R C I A L P R I O R I T I E S
• but if it was going to work for
us, it needed to be a “purely”
fun, commercial game
• so we switched on our
commercial developer brains:
bomb tiles, gold bonuses,
snappy chatter replies,
iWatch-style wiggling tiles,
leaderboards - not things you
would prioritize for learning

39. – F R I E N D - O F - B E TA T E S T E R , ‘ A L L I S E ’
“Tell your friends thanks for Dokudo. I played it all
the way across Canada.”
results so far?
and…
“Ok so I had a stroke when I had alex 11 years ago.
I have aphasia ( difficulty speaking) my speech
therapist has me doing crosswords and word finds.
This game hasn't left my fingers for 4 hours now!
Lol.”

40. – F R I E N D - O F - B E TA T E S T E R , ‘ A L L I S E ’
“Tell your friends thanks for Dokudo. I played it all
the way across Canada.”
results so far?
and…
“Ok so I had a stroke when I had alex 11 years ago.
I have aphasia ( difficulty speaking) my speech
therapist has me doing crosswords and word finds.
This game hasn't left my fingers for 4 hours now!
Lol.”
…a commercial, ‘addictive’ game
…that’s helping a stroke
victim recover language,
lol

41. I T ’ S E A R LY Y E T, B U T
• we started with a competency, which was
also a leverage point, which was an idea
that hadn’t been done in a commercial
game before
• we conceived of a mechanic that was
intrinsically performative of what we
wanted to create: word creativity with
graphemes
• we built that game commercially, to be
fun-first, to stand on its own*
• …and I think that’s glasslab 2.0 (for me,
anyway)
* with literally 0 budget - full indie

42. K T H X B Y E
erin@makingwonder.com
questions?

43. A B O U T S E N S E O F W O N D E R
A P P E N D I X :

44. C O M PA N Y C O N C E P T
• bringing the power of
asymmetrical free-to-play player
ecosystems to education
• premium consumer experiences
subsidize under-resourced school
access
Founded in 2012, Sense of Wonder is a fun-first game studio with a social
mission. Dedicated to making “smart fun” games in the tradition of Sid
Meier and Will Wright, Sense of Wonder combines an indie art sensibility
with engaging mechanics that bring out the natural wonder in the world.
Founded by online game veterans, Sense of Wonder’s mobile and tablet
games use “true social” mechanics that meaningfully connect players, and
employ compassionate player policies to ensure that customer loyalty is not
just earned, but kept over the years.

45. T E A M
Erin Hoffman
Cofounder & Chief Designer
Michal Todorovic
Cofounder & Chief Technologist
founder @ Terra Nova, Warm & Fuzzy Logic
engineering director @ Zynga 2009-2012
27 years making games
published fantasy author
lead designer @ Zynga 2010-2011
17 years making games
…plus our expandable bench of on-call ninjas; staff profiles available
upon request