Learning games have created a buzz in the training industry. Games promise a motivating and fun learning environment, but do these games lead to better learning? During this webinar, the Doorways to Dreams Fund (D2D), Enspire Learning, and the Skillpoint Alliance will discuss the development of the casual video game, “Celebrity Calamity.” While managing the finances of spendthrift celebrity, this new game teaches low- to moderate-income women the basics of credit and debit cards. For many in the target audience, personal finance education can be difficult, scary, and boring. Keeping players motivated and engaged was important, though only half the battle. For the game to be considered successful, players also had to emerge from the experience with the confidence -- and knowledge -- to manage their credit and debit card accounts. To facilitate this, D2D and Enspire conducted rigorous user experience testing during the iterative production process. Later, efficacy testing was used to determine knowledge and confidence changes as a result of the game. The preliminary results? Tests with small samples showed a 50-70% improvement in financial knowledge and a 15-30% confidence increase in key areas.

1. Building Better Learning Games:
Leveraging Game Design and User Testing for Results
April 9, 2009
Austin, TX
© 2009 Enspire Learning and D2D Fund, Inc.

2. Today’s Agenda
Introductions
Casual video games and learning needs
Case Study: “Celebrity Calamity”
User-testing and Game Development: Lessons Learned
Questions and Answers
-2-
© 2009 Enspire Learning and D2D Fund, Inc.

3. Introductions
Ben Katz, Enspire Learning, Lead Game Developer
Nick Maynard, D2D Fund, Director of Innovation
Kristy Bowden, Skillpoint Alliance, Director of Digital Media Council
-3-
© 2009 Enspire Learning and D2D Fund, Inc.

4. Enspire Learning
Founded in 2001 by Bjorn Billhardt
and based in Austin, Texas, Enspire
creates:
Learning Games
Simulations
E-Learning Solutions
“Rise of the Shadow Specters”
Simulation
Simulation E-Learning
-4-
© 2009 Enspire Learning and D2D Fund, Inc.

5. Doorways to Dreams (D2D)
Doorways to Dreams Fund
(www.d2dfund.org) was co-founded in
2000 by Harvard Business School
Professor and Associate Dean, Peter
Tufano; D2D is
A non-profit 501(c)3 headquartered in
Roxbury, Massachusetts,
Whose mission is to increase savings and
asset-building for poor and working poor
Americans,
And our corporate analogues are an out-
sourced:
– R&D lab
– New products group
– Product management team
-5-
© 2009 Enspire Learning and D2D Fund, Inc.

6. Skillpoint Alliance – Digital Media Council
Skillpoint Alliance is a 501(c)3 nonprofit organization that builds partnerships
with industry, education and community to support economic growth in Central
Texas.
Skillpoint’s Digital Media Council focuses on the digital media and creative
technology fields in the region, bringing together industry executives and
educational and community leaders to engage in joint problem-solving on
challenging local workforce and education issues.
-6-
© 2009 Enspire Learning and D2D Fund, Inc.

7. Today’s Agenda
Introductions
Casual video games and learning needs
Case Study: “Celebrity Calamity”
User-testing and Game Development: Lessons Learned
Questions and Answers
-7-
© 2009 Enspire Learning and D2D Fund, Inc.

8. Games at 50,000 Feet
While this is not new to many (nor comprehensive), games come in a variety of
shapes and sizes:
Console Games: Halo (XBOX), Super Smash Brothers (Wii)
MMORPG (Massively Multi-player Online Role Playing Game): World of Warcraft
(Adults), Club Penguin (Kids)
Simulations: Flight Simulator, The SIMS (a strategic life simulation game)
ARGs (Alternate Reality Games): World Without Oil
– From Wikipedia: An interactive narrative that uses the real world as a platform, often involving
multiple media and game elements, to tell a story that may be affected by participants' ideas
or actions.
But there is another whole genre….
-8-
© 2009 Enspire Learning and D2D Fund, Inc.

9. Casual Games
Do you play any of these games?
-9-
© 2009 Enspire Learning and D2D Fund, Inc.

10. Casual Games
Bejeweled
Tetris
Solitaire
- 10 -
© 2009 Enspire Learning and D2D Fund, Inc.

11. Casual Game Demographics
In 2007, 61% of online game play was either
puzzle/board/game show/trivia/card games or downloadable
version of games like Bejeweled and Diner Dash
- 11 -
© 2009 Enspire Learning and D2D Fund, Inc.

12. Casual Game Demographics
Who is playing casual games?
Gender
– 63% Women
– 37% Men
Age
– 71% are 25-54
• 21% are 25-34
– 17% are 55+
Families
– 46% have children
Experience online
– 90% have been on the Web longer than
three years
- 12 -
© 2009 Enspire Learning and D2D Fund, Inc.

13. Casual Game Design
Casual video games share these design principals:
Low barriers to entry
– Very few screens to go through before start
– Few or “as you play” instructions
Forgiving
– No punishment if you do something wrong
Skywire 2
– Rewards error, oftentimes
Short play times
– During a coffee break, can have meaningful play for 10 minutes
Highly re-playable
Depth of game play is revealed gradually (“accessible” to everyone)
Themes are often non-violent
- 13 -
© 2009 Enspire Learning and D2D Fund, Inc.

14. Casual Game Development
Casual games make it easy to: Diner Dash
Build a prototype and iterate quickly
and efficiently
Closely match game mechanics to
key learning objectives
Deploy games online in Flash (and
eventually on web portals such as
Kongregate)
- 14 -
© 2009 Enspire Learning and D2D Fund, Inc.

15. Today’s Agenda
Introductions
Casual video games and learning needs
Case Study: “Celebrity Calamity”
User-testing and Game Development: Lessons Learned
Questions and Answers
- 15 -
© 2009 Enspire Learning and D2D Fund, Inc.

16. Why are we here?
- 16 -
© 2009 Enspire Learning and D2D Fund, Inc.

17. Financial Entertainment
“D2D’s vision is financial entertainment. Taking cues from business and
entertainment, we need to work with and for consumers in the development of
engaging new media that teaches them how to better manage their money.”
As a starting point, we envision a library
of casual video games, each teaching
simple financial lessons; these include:
Use of credit and debit cards;
Personal budgeting;
Basic calculations, like compound interest;
Saving for retirement;
Student and other loans; and
Awareness of expensive pitfalls (i.e., pay-
day lending, check cashing)
- 17 -
© 2009 Enspire Learning and D2D Fund, Inc.

18. Why financial literacy casual games?
To date, efforts in this space have
been unsuccessful at finding the right
balance between fun and education:
Very complex simulations that can
seem like Quicken in a video game
Flashy graphics and audio
surrounding a multi-choice quiz type
presentation of facts
Brand marketing packaged as an
video game
- 18 -
© 2009 Enspire Learning and D2D Fund, Inc.

19. First Prototype Game: Celebrity Calamity
While focused on fun, the game’s explicit
learning objectives include:
Paying more than the minimum credit
card payment;
Minimizing credit card finance charges;
Avoiding all fees including bank
overdraft, credit card late payment, and
credit card over-limit; and Game Trailer:
www.youtube.com/watch?v=oX2tr4RHvYM
Making good annual percentage rate
(APR) choices.
Prototype Game:
The game also includes a number of
implicit learning objectives, such as www.celebritycalamity.com/
raising awareness of spending behavior
and the value of saving money.
- 19 -
© 2009 Enspire Learning and D2D Fund, Inc.

20. Celebrity Calamity: Lessons Learned
Know your audience.
Games are a popular form of digital media.
Listening to people’s needs and preferences is important.
Evaluation is important.
- 20 -
© 2009 Enspire Learning and D2D Fund, Inc.

21. Know your Audience.
We are creating games for a This segment faces a number of
financially vulnerable market: important challenges:
Low to moderate income females Very low levels of financial literacy
18-32 years old Peer effects do not work in their
favor
Single or Married with children
Little or no savings
Household income less than
$40,000 Limited opportunities to receive
financial education
Education of at least high school or
GED
Ethnically diverse
- 21 -
© 2009 Enspire Learning and D2D Fund, Inc.

22. Games are a very popular form of digital media.
Digital media offers the potential to
marry fun and education:
72% of Americans are playing video
games;
The rates of play are highest among
those under the age of 35;
Casual video games are the fastest
growing segment in the game
industry; and
The casual segment is dominated by
female players
- 22 -
© 2009 Enspire Learning and D2D Fund, Inc.

23. Listening to people’s needs and preferences is important.
Celebrity Calamity was created in partnership with users:
Game played and tested at three key development milestones with audience;
Individual, focus-group, and observational feedback;
Extension of the game development process timeline to integrate feedback into design.
- 23 -
© 2009 Enspire Learning and D2D Fund, Inc.

24. Celebrity Calamity Game Development Process
Test Test Test
First Alpha Final
Beta
Playable
At each development milestone, :
Fun. Are players enjoying the game? Do they want to continue playing (if the game was
fully developed)? What components of play are most enjoyable?
Learning needs. What do players know about the teaching topics? What don’t players
know? What aspects of the game connect with real life?
Assessment. Are players engaged? What level of enthusiasm due they show for the
content? Do we observe increases in self-confidence?
- 24 -
© 2009 Enspire Learning and D2D Fund, Inc.

25. Game Development Process: Additional Benefits
This development partnership also:
Brings together digital media
companies with a financially
vulnerable audience;
Exposes low-to-moderate income
women to the digital media industry;
Raises awareness about the
importance of digital media’s
potential impact on societal issues;
and
Highlights the digital media industry.
- 25 -
© 2009 Enspire Learning and D2D Fund, Inc.

26. Evaluation is Important.
D2D has completed preliminary testing
of Celebrity Calamity, and we are
pursuing more rigorous evaluation
given the positive initial results
Early evaluation was incorporated in
the product development process at
Beta
Preliminary formal testing has been
completed with 44 LMI women in three
cities: Austin, TX, Tulsa, OK, and
Providence, RI
Additional play-testing has been
completed with high school students in
Dorchester, MA and an older Spanish-
speaking audience in Lawrence, MA
- 26 -
© 2009 Enspire Learning and D2D Fund, Inc.

27. Preliminary Game Efficacy Testing: Self-Confidence
Table 1: Self-Confidence (Pre and Post, % Change)
Rate your degree of confidence in the following: Pre Post %
Change bad spending habits 3.50 3.89 11%
Identify needs and wants separately. 4.02 4.30 7%
Avoid impulse and emotional spending. 3.23 3.93 22%
Pay off new charges on credit cards every month. 3.16 3.84 21%
Pay more than the minimum amount on credit card bills. 3.16 4.09 29%
Avoid late fees, over-limit fees, and overdraft fees. 3.23 4.05 25%
Save money regularly. 3.14 4.14 32%
Manage my finances. 3.57 4.09 15%
Perform assigned job-related tasks. 4.23 4.43 5%
Follow directions from my boss/supervisor. 4.66 4.61 -1%
- 27 -
© 2009 Enspire Learning and D2D Fund, Inc.

28. Preliminary Game Efficacy Testing: Knowledge
Table 2: Knowledge (Pre and Post, % Change)
Knowledge Area: Pre Post %
Annual Percentage Rate (APR) 55% 86% 58%
Finance Charges 48% 82% 71%
Pay More Than the Minimum 82% 93% 14%
Interest Compounding 61% 64% 4%
- 28 -
© 2009 Enspire Learning and D2D Fund, Inc.

29. Qualitative Feedback from the Audience
Enthusiasm: “Can I have a copy of
the game? Where can I buy it?”
Engagement: “I’ve got the hang of it
now, I can do this.”
Education: “I’ve had credit cards for a
while now, but I did not know what
annual percentage rate, APR, was,
until now.”
Empowerment: “It helped me to
better understand how to manage
money.”
- 29 -
© 2009 Enspire Learning and D2D Fund, Inc.

30. Celebrity Calamity: Lessons Learned
Know your audience.
– While casual video games are quite popular among women, in general, identifying a
specific demographic allows for focused innovation.
Games are a popular form of digital media.
– Despite the “Digital Divide,” lower-income young women have grown up playing video
games are actively engaged with digital media.
Listening to people’s needs and preferences is important.
– By incorporating the end-user into game development, D2D has created a
“participatory learning” platform for financial education.
Evaluation is important.
– Assessment and feedback are built in throughout the product development processes
as well as more formally once game development is complete.
- 30 -
© 2009 Enspire Learning and D2D Fund, Inc.

31. Celebrity Calamity Next Steps
More rigorous evaluation of Celebrity
1.
Calamity;
Making additional casual games
2.
(second game in development); and
Testing distribution strategies along
3.
with associated social marketing
techniques; target channels include:
– Employers
– Community Colleges
– Grassroots Organizations (non-profits)
– Financial Institutions
- 31 -
© 2009 Enspire Learning and D2D Fund, Inc.

32. Testing Partnerships
If you want to join a growing list of national employers and organizations testing
distribution of Celebrity Calamity, please contact Nick Maynard,
nmaynard@d2dfund.org
We are interested in game deployments that would help us learn the following:
• How many employees/individuals play the game? For how long do they play?
What do we know about them demographically (e.g., age, gender)? Where do
they play the game? Do other members of their household play the game?
• Which marketing approaches work best? Which frame is most effective for
driving play? Does content associated with game characters engage the
audience?
• Are there other educational materials that can be combined with the game to
improve impact with specific audiences?
- 32 -
© 2009 Enspire Learning and D2D Fund, Inc.

33. Today’s Agenda
Introductions
Casual video games and learning needs
Case Study: “Celebrity Calamity”
User-testing and Game Development: Lessons Learned
Questions and Answers
- 33 -
© 2009 Enspire Learning and D2D Fund, Inc.

34. User-Testing: Lessons Learned
Test early, test often
– Can’t make game in a vacuum
– Test at key deliverables, and then iterate
Usability testing first, then efficacy testing later on
– Early prototypes more focused on creating best user experience
– Focus on knowledge and confidence changes after usability of game is assured
Collect as much good data as possible
– Likert scale based confidence questions
– Knowledge questions
– In-game data collection for future analysis
– Avoid focus groups until end of session
- 34 -
© 2009 Enspire Learning and D2D Fund, Inc.

35. Game Design: Lessons Learned
Games you like aren’t necessarily games your target demographic likes
– Game designers aren’t like everyone else. They’re weird.
– Interface elements we take for granted aren’t understood by everyone (ex. Click to
Continue, Slider)
Prototype early and iterate often
– Avoid heavy multimedia development until later on
– Game appearance isn’t as important as game mechanics, especially at an early stage
Game goals must be immediately apparent to players
– This is not the place for subtlety
– Win and lose states must be reasonable and fair
– Learning objectives must be closely matched to success in gameplay
- 35 -
© 2009 Enspire Learning and D2D Fund, Inc.

36. Education: Lessons Learned
Discipline in the scope of teaching content
– Fun can quickly be overwhelmed by rush to include teaching points
– If teaching points are not relevant to game play, players will miss them
Repetition, repetition, repetition
– The second celebrity in Celebrity Calamity and APR
• Character talks about APR
• APR changes every round
• “Calamities” are tied closely to APR impacts (e.g., cash advances)
• APR offers dropped with the money
– Multiple touch points in the game design
Players can learn from failure
– Games allow for changing strategy and trying new things
– Scoring must reinforce learning points clearly
- 36 -
© 2009 Enspire Learning and D2D Fund, Inc.

37. Question and Answer
- 37 -
© 2009 Enspire Learning and D2D Fund, Inc.

38. Presenter Contact Information
Ben Katz, Enspire Learning: ben.katz@enspire.com
Nick Maynard, D2D Fund: nmaynard@d2dfund.org
Kristy Bowden, Skillpoint Alliance: kbowden@skillpointalliance.org
- 38 -
© 2009 Enspire Learning and D2D Fund, Inc.