Presentation given at SPBT Conference June 2013.

1. Lessons from Harvard:
Using Gamification to Juice
Your Sales Training
B. Price Kerfoot, MD EdM
VA Boston Healthcare System
Harvard Medical School

2. Conflict of Interest Disclosure
• Harvard submitted patent on spaced education
methodology in 2006.
• Harvard launched a start-up company (Qstream,
www.qstream.com) which hosts this methodology
outside of the Harvard firewalls.
• As the “inventor”, I am an equity owner and director
of this company.

3. Structure of Talk
• What is a game?
• My (strange) research trajectory
• Elements of game mechanics
• Rewards
• Leaderboards
• Teams
• Examples of games in sales training

4. • 2013 Horizon Report on Educational Technology:
‘game-based learning’ as one of six emerging technologies
likely to have a large impact on education over the coming 2-3
years.1
• What is a game?
Games as Educational Tools
1
Johnson et al. (2013). The NMC Horizon Report: 2013 Higher
Education Edition. Austin, Texas: The New Media Consortium.
http://www.nmc.org/pdf/2013-horizon-report-HE.pdf

5. What is a Game? Examples…

6. Salen & Zimmerman. Rules of Play: Game Design Fundamentals. Cambridge Mass: The MIT Press. 2004.

7. Working Definition of an Educational Game:
“an active, outcome-oriented learning experience
that involves focused decision-making
and proceeds according to rules”
Games as Educational Tools
• Active
• Outcome-oriented
• Learning experience
• Focused decision-making
• Proceeds according to rules

11. .

13. .

14. .

15. • Guilds: Blood legion, Zenith clan
• Quests: Collection quests, Kill quests
• Levels:
• 1st
-level Warlock
• 6th
-level Rogue
• 13th
-level Death Knight
• 20th
-level Shaman
Game:
World of Warcraft

16. • Guilds: Blood legion, Zenith clan
• Quests: Collection quests, Kill quests
• Levels:
• 1st
-level Warlock
• 6th
-level Rogue
• 13th
-level Death Knight
• 20th
-level Shaman
Game:
World of Warcraft
Crazy ?

17. • Guilds: SBPT
• Quests: Making your Q2 numbers
• Levels:
• Salesperson of the year
• District manager
• Regional manager
• VP of sales
Games are everywhere!
Game:
Selling Pharmaceuticals

18. “90% of people
attending Gamblers
Anonymous meetings
in Las Vegas play
only (slot) machines.”
-- Natasha Dow Schull, anthropology professor at MIT

19. How can we build a slot machine
for learning ?

20. Multi-institutional randomized controlled trial of 351 medical
students at 4 northeastern medical schools
Acad Med. 2006;81(3):224-30.

21. Retention of BLS and ACLS Training Among Nurses
Resusc 2008; 78: 59-65
n=133

22. Lammers (Michigan State Univ) Acad Emerg Med 2008; 15: 1181-89

23. Spaced Education Delivery System:
harnesses both the spacing & testing effects

24. Spaced Education Delivery System

25. VA PSA Screening Spaced Education Trial

26. VA PSA Screening Spaced Education Trial

27. VA PSA Screening Spaced Education Trial

28. It Works!
• Increases knowledge & retention
- Medical Education 2007: 41:23-31 --- UGME
- Journal of General Internal Medicine 2008; 23(7):973-8 --- UGME
- Academic Medicine 2011 (in press) --- UGME
- Journal of Urology 2007; 177, 1481-1487 --- GME
– Journal of Urology 2009; 181, 2671-2673 --- GME
– Journal of the American College of Surgeons 2010; 211: 331-72673 --- GME
– Annals of Surgery 2009; 249: 744–749 --- CME
• Improves self-assessment of knowledge
– American Journal of Surgery 2009; 197(1):89-95
• Changes behavior
– American Journal of Surgery 2009: 197(2), 252-257
– American Journal of Preventive Medicine 2010; 39: 472-8
• Is well-accepted by learners
- Demonstrated in all trials to date

29. VA PSA Screening Spaced Education Trial

30. How can we build a slot machine for learning ?
Game Mechanics:
► Adaptive algorithm: Qs re-sent at variable
intervals if answered correctly vs incorrectly.
► Progression dynamic: Physicians retired Qs by
answering each correctly twice-in-a-row.
► Appointment dynamic: Qs expired if not
answered on time.
► Competition was fostered by posting relative
performance among physicians.
► Awards: CME credit given based on
performance.

31. 2012
2012

32. 2012

33. Global Guidelines Dissemination
Hematuria, priapism, staghorn calculi, infertility, & antibiotic use
Annals of Surgery 2012 Jul;256(1):33-8

34. Global Guidelines Dissemination
Annals of Surgery 2012 Jul;256(1):33-8

36. Spaced Education
Delivery System

41. QstreamOnline Game
3N 2NSICU
Rank Team Ave. Score
1 3N 4083.7
2 SICU 3669.4
3 2N 3581.4
4 A1-MSDU 3425.8
5 CCU 3210.9
6 MICU 3206.0
7 AG 2906.2
8 A2 2493.1
9 PACU-PCU 2153.9
10 2S 1954.3
2S AGA2
Qstream Online Game
GO Team GO!!!
Nurses check your email every,
Tuesday and Thursday
got Qstream?

44. Cost & Utilization Outcomes

47. Structure of Talk
• What is a game?
• My (strange) research trajectory
• Elements of game mechanics
• Rewards
• Leaderboards
• Teams
• Examples of games in sales training


48. Reward Hierarchy
S – status
A – access
P – power
S – stuff
“SAPS”

49. Saeed
Abdul
Ghafour
Khouri
Status

50. Status

51. To become a Gilt Noir member, you must
spend at least $10,000 on Gilt.com a year
Lets the top 1% of users preview sales
15 minutes before they go live online
Access

52. Access

53. Power

54. Impact of Incentives -- The Candle Problem
“Fix the candle to the wall so that the wax doesn’t drip on the table”
Glucksburg 1962, cited in Drive by Daniel Pink (2009)
Study:
Top - $25
Top 25% - $5
v.
No reward

56. The Candle Problem
“Fix the candle to the wall so that the wax doesn’t drip on the table”
Glucksburg 1962, cited in Drive by Daniel Pink (2009)
Study:
Top - $25
Top 25% - $5
Vs.
No reward

57. The Candle Problem – Solved
“Fix the candle to the wall so that the wax doesn’t drip on the table”
Glucksburg 1962, cited in Drive by Daniel Pink (2009)
Study:
Top - $25
Top 25% - $5
…3.5 minutes longer
v.
No reward

58. The Candle Problem
“Fix the candle to the wall so that the wax doesn’t drip on the table”
Glucksburg 1962, cited in Drive by Daniel Pink (2009)
Interesting…but how does this relate to my sales force?
“Functional fixedness”

59. Designing Effective Sticks and Carrots:
• Several studies show that large tangible contingent rewards
can extinguish intrinsic motivation & limit creativity
….in certain situations
• Ideal: Align extrinsic motivators with intrinsic motivations
• Be creative !

60. Structure of Talk
• What is a game?
• My (strange) research trajectory
• Elements of game mechanics
• Rewards
• Leaderboards
• Teams
• Example of games in sales training


61. Leaderboards

63. Alias…or not
Structure matters!

64. Structure of Talk
• What is a game?
• My (strange) research trajectory
• Elements of game mechanics
• Rewards
• Leaderboards
• Teams
• Examples of games in sales training


65. Ann Intern Med. 2013;158:505-514.

66. Team Size
Matters!

67. r = -0.68
p < 0.05
Psychol Sci 2009; 20(7):871-877.

68. Structure of Talk
• What is a game?
• My (strange) research trajectory
• Elements of game mechanics
• Rewards
• Leaderboards
• Teams
• Examples of games in sales training

69. EndoWrist® One™
Vessel Sealer

75. mLearnCon 2013 Mobile Learning Conference, June 18-20, San Jose CA

78. Spaced Educ Group Feedback:
"Convenient, timely, effective"
"Learned material without
interrupting my day"
"Nothing beats being able to
answer questions and getting
immediate feedback"
"Qstream keeps you engaged
and works well for new devices
such as vessel sealer.”
“An effective way to ensure the
data sticks”
No phone calls!

79. Game Mechanics Can Juice Your Sales Training