The document discusses plans to assess the effects of micro-credentials/badges on game completion and engagement for the HuskyHunt game. The assessment will use quantitative and qualitative measures to evaluate completion rates, engagement, skills learned, and shifts in attitudes from players in a version of the game that offers badges compared to a version without badges. The game will be modified to collect additional usage data and will provide opportunities to earn badges linked to completing modules or tasks. Surveys and interviews will also help assess the influence of badges and learning outcomes.

1. The Practicum
My intention with the practicum is to leverage the existing HuskyHunt framework and
measure the effects of micro-credentials/badges on game completion and player engagement.
The assessment will be done using the quantitative and qualitative assessments described below.
Analysis and Design of Technologies, Lesson content and Pedagogy
HuskyHunt
The university recognizes that its students do not have sufficient understanding of the
personal security and behavioral risks associated with their activities on the Internet (e.g.,
copyright infringement, phishing, electronic banking security). There is also concern
regarding the students general apathy related to personal privacy. Because mandatory
training was discussed but dismissed, HuskyHunt was created to provide a learning format
that students would opt into.
HuskyHunt is a web based application designed with four specific goals:
 Teach students practical lessons about Internet and information security awareness
topics that they can relate to personally
University of Connecticut
Formative Assessment
of HuskyHunt
Pufahl, Jason
8-12-2014

2. Formative Assessmentof HuskyHunt
1
 Utilize gaming mechanics to increase engagement and participation
 Leverage Social media, specifically Facebook and Twitter to reinforce learning that
occurred in the game modules.
 Leverage social media to increase the likelihood that UConn students not
participating in HuskyHunt are exposed to the security training.
HuskyHunt is designed to be delivered at a large scale and has no technical limitations to
the number of individuals that can participate in an available and active course. The
framework aligns most closely to an xMOOC. An xMOOC is geared towards an individual
seeking an achievement, is likely offered in addition to other complementary education and is
designed to be repeated (“elearnspace › MOOCs are really a platform,” n.d.).
The application was originally designed with no specific pedagogy in mind, but upon
reflection I feel the online content and assessment framework most closely resembles an
objectivist learning theory (Duffy & Jonassen, 1992). HuskyHunt was developed and
implemented to deliver “the entities, relations and attributes that the learner must ‘know’”
(Duffy & Jonassen, 1992 location 192). It also shares similarities with a current standards
based curriculum concepts (Carr-Chellman, 2011) leveraging directed instruction and
assessments. HuskyHunt establishes clearly communicated learning objectives and leverages
online multiple choice assessments designed to test specifically what was taught in the
lesson.
There game has incorporated, inadvertently, some of Gagne’s 9 events of instruction
(Pastula, 1999):
 Marketing and prizes were used to gain potential players attention
 Materials to be learned were presented in chunks
 Learning was assessed
Recognizing that the primary online teaching module of HuskyHunt does not actively
engage students in the learning process, the application has a secondary, real world module
that leverages the concept of practice fields (Jonassen & Land, 2012). The secondary module
is designed to engage players in practical activities that reinforce concepts taught during the
online portion of the game.
The application has been marketed to students primarily as a game and secondarily as a
security awareness education tool. The distinction was made with the hope that playing a
game containing tangible prizes (e.g. cash rewards, coupons at local merchants, swag) would
incentivize students to engage in security awareness training that isn’t mandatory. We
originally underestimated the students desire to learn and over the last few years have
modified the marketing efforts to more clearly position the game as a learning opportunity
with prizes.
The application employs three basic game mechanics:
 There are Rules

3. Formative Assessmentof HuskyHunt
2
 There is a scoring system
 There are rewards throughout and during the game for participants and winners
Learning modules are presented and assessed using multiple choice questions and
answers. As students complete modules they earn points which ultimately positions them to
win the grand cash prize. Players are also able to earn points by participating in secondary
modules designed to support the learning objective of the primary module. Player motivation
occurs primarily through a points system designed around module completion and secondary
activities participation. Additional in game prizes such as merchant coupons are also
available periodically throughout the game.
Social Media is used in two ways:
 Allow the game master to communicate game events to player.
 Allow players to share security awareness information to their
friends/followers.
All announcements utilize the hashtag “#HuskyHunt” for consistency.
Micro-Credentials
Micro-Credentials, also known as digital badges, are designed to provide confirmation
for the successful achievement of particular activities. Digital badges are conceptually similar
to the physical badges (i.e., insignia) used in the military. Digital badges serve a variety of
functions, including (“Digital badges - Wikipedia, the free encyclopedia,” n.d.), (“About |
Open Badges,” n.d.)
 To provide a reward
 To convey some relevant information
 To establish a goal
 To identify someone as a member of a group
 To provide status
Badges will be added to the existing game framework as a mechanism to provide
increased incentives for players to participate in and complete the game.
Expected Outcomes and Objectives
The general instructional objective of the project will be to determine the effect that the
opportunity to earn and collect digital badges has on participation and completion rates.
The information gained from the study will be used to infer whether or not digital badges can
improve access to information. However, it will not specifically assess whether or not learning
occurred. It is expected that the players engaged in the application version that leverages badges

4. Formative Assessmentof HuskyHunt
3
will progress further and more quickly through the game than those players not receiving badges.
It is also expected that the players receiving badges indicate that they enjoy the game more.
Digital Badge Implementation
Clear, concise badge design will be essential for establishing motivators that sufficiently
contribute to completion and engagement. Badge design has not yet occurred, but I am initially
considering the following badges (Badge titles are currently descriptive only):
 Fastest to Answer Primary Module:
Awarded to the first Player that completes a primary learning module. There will be an
opportunity to earn this in each primary learning module.
 Fastest to Answer Secondary Module:
Awarded to the first Player that completes a secondary learning module. There will be an
opportunity to earn this in each secondary learning module.
 Security Ninja:
Awarded to anyone completing the primary and all secondary daily modules.
 Fastest to Answer ALL Primary Modules:
Awarded to the individual that was quickest, on average, to complete all Primary
modules.
 Most recruits:
Awarded to the player that recruits the most players to join.
 Gone Phishing:
Awarded to players passing the Phishing module the first time.
 Spread the word:
Awarded to players posting game provided security messages to Facebook/Twitter.
Game Implementation
The current application and gameplay will be modified to ensure that the data collected
will be useful for analysis. The following are key decision points:
 The game will be run as online-only, omitting the real world scavenger hunt.
 The game will permit user created accounts and no longer rely solely on NetID for
access. This change should improve player demographics by enabling players to be
recruited both internally and externally to UConn.

5. Formative Assessmentof HuskyHunt
4
 The game will have two versions, one with badges and scoring and the other with scoring
only.
 The games will be limited to one week, but two game sessions will be run. This will
permit the collection of data over multiple sessions and is expected to provide more
reliable results.
 The application will be designed to log additional data points to improve analytical
options. It is currently expected that the following data will be collected: User
demographic information, Time spent on each page, page transition order, assessment
statistics (e.g., success, failure, and attempts), social media participation. It is likely more
data will collected as assessments are refined.
 The game will include a video introduction by me. This will provide a human element to
the game and hopefully establish trust in the game, the creator, and the purpose.
The most significant challenge will be recruiting players. Unfortunately, I believe that
utilizing the more common method of announcing an incentive driven research study participant
opportunity (e.g., via listserv, daily digest, posters) would likely produce players that felt some
requirement to complete the game. The major consideration is collecting participants that want to
play and do not feel that participation or game completion is in any way mandatory. I am
currently evaluating ways to leverage security contacts across Connecticut Higher Education to
assist in identifying potential new player pools.
Once potential players are identified, they will be sent instructions describing the game,
the learning objectives, the account creation process, and the game start and end dates. As
players sign up, they will be randomly added to game control groups. One control group will
have opportunities to earn both points and digital badges and one control group will only earn
points. Players will have an opportunity to recruit other players and will receive points and/or
badges for their recruiting effort. Players will only be able to recruit until the time the game
begins.
The game will be run with a primary daily lesson and associated assessment. The
assessments are expected to remain multiple choice. There will be multiple secondary lessons
that occur each day. These lessons will be offered at random times during the day and will likely
be available only for short intervals. Both primary and secondary lessons will provide
opportunities for players to earn points and badges if applicable. It is expected that there will be
time limits to earn maximum points and that point values will decay to an established minimum
greater than zero as time to completion increases. It is important that players always have the
opportunity to earn some points so there is some incentive to complete each module.
As points and badges are earned they will be displayed on a leaderboard. All players will
be able to view their current position in relationship to other player at any time.
The game will end with a final assessment likely to contain:
 Multiple choice assessment utilizing questions drawn from each module assessment
 Multiple choice questions relating to each module, but not drawn directly from
module assessments

6. Formative Assessmentof HuskyHunt
5
 Practical test to see if players respond to a phishing message
 Practical test to see if players provide username and password information when
asked for it
Players will be presented with a short optional survey at the end of the game and asked if they
would be willing to participate in a short interview.
Evaluation
Assessments will be developed that produce quantitative statistics to measure completion
rate, engagement, and skills achieved. Qualitative data will be gathered and used to measure
proficiency in new technology literacies and attitudes towards digital badges and personal
information security concepts.
Completion Rate
Completion rate will be evaluated using the following statistics:
 % of players completing each module
 % of players remaining after each module
 % of players completing all modules
 % of players abandoning game at some point
 % of players abandoning by module
Engagement
Engagement rate will be evaluated using the following statistics:
 Average elapsed time between lesson release and assessment completion
 % players posting to social media
 Recruitment of new players
 % of players completing post-game survey
Skills Learned
 Average times player required to answer an assessment question correctly
 Average times player required to answer final assessment question correctly
 % of people responding to spam email test
 % of people providing username and password
Attitudes Shift and New Learning Literacies
A post-game survey will be used to obtain a greater understanding of the following
information:
 Player’s thoughts and knowledge about Internet and information security before and
after the game. A pre-game interview will not be conducted, so player attitude
changes will need to be assessed only through a post-game interview.

7. Formative Assessmentof HuskyHunt
6
 Player’s thoughts on how badges influenced their desire to play the game.
 Players attitudes towards the technologies and concepts used in the game:
o Twitter/Facebook
o Badges
o Mobile application access
o Points and scoring
o Google Hangouts
o Google Forms or Socrative
The post-game survey will also be used to identify players that would be willing discuss,
via Google Hangouts, their perspective on the game and its technologies. These informal
interviews will be used as supporting data.
About | Open Badges. (n.d.). Retrieved August 9, 2014, from http://www.openbadges.org/about/
Carr-Chellman, A. A. (2011). Instructional design for teachers: improving classroom practice.
New York: Routledge.
Digital badges - Wikipedia, the free encyclopedia. (n.d.). Retrieved August 9, 2014, from
http://en.wikipedia.org/wiki/Digital_badges
Duffy, T. M., & Jonassen, D. H. (Eds.). (1992). Constructivism and the technology of
instruction: a conversation. Hillsdale, N.J: Lawrence Erlbaum Associates Publishers.
elearnspace › MOOCs are really a platform. (n.d.). Retrieved from
http://www.elearnspace.org/blog/2012/07/25/moocs-are-really-a-platform/
Jonassen, D. H., & Land, S. M. (Eds.). (2012). Theoretical foundations of learning environments
(2nd ed.). New York: Routledge.

8. Formative Assessmentof HuskyHunt
7
Pastula, P. (1999). Applying Learning Theories to Online Instructional Design. Pastula.com.
Retrieved August 9, 2014, from
http://www.patsula.com/usefo/webbasedlearning/tutorial1/learning_theories_full_version
.html