NISO Virtual Conference: Expanding the Assessment Toolbox: Blending the Old and New Assessment Practices Assessing Game-Based Library Initiatives Kyle Felker, Digital Initiatives Librarian, Grand Valley State University Libraries

1. ASSESSING GAME-BASED
PROGRAMMING
Issues, challenges, and models to follow

2. WHO I AM
Kyle Felker
GrandValley State University
Libraries
Digital Initiatives Librarian
Investigates, prototypes,
integrates
felkerk@gvsu.edu, @gwydion9

3. STRUCTURE OFTHISTALK
What games can do for us
Basic assessment principles
Unique Challenges and approaches
LibraryQuest assessment program

4. WHY GAMES?
Greater, lengthier, or more focused engagement with facilities,
staff, services, etc.
Reaching one or more learning objectives

5. GENERAL PRINCIPLE : SET CLEAR
AND MEASURABLE GOALS
Engagement based-goal: Increase participation in our summer
reading program by at least 50%
Learning goal:To teach incoming undergraduates how to locate
books using the online catalog and how to find them on the shelf.
Skills-based vs Conceptual

6. GENERAL PRINCIPLE: CONTEXTUALIZE
AND HARMONIZE GOALS
Should be in accord with institutional goals and objectives
Should not duplicate other programs
Should take into account national and institutional standards,
frameworks, etc.

7. EVALUATIONVS.ASSESSMENT
Evaluation: How can we improve this?
Assessment: Did this do what it was supposed to do?
Evaluation often easier, mistaken for assessment
Assessment often requires more effort and thought

8. GENERAL PRINCIPLE: BUILD ASSESSMENT
INTO PROJECTS AT INCEPTION
Consider assessment plan as part of overall plan
Design, schedule, and plan opportunities for both evaluation and
assessment
Secure resources such as staff time and money
Initial pilot: evaluation, subsequent pilot: assessment

9. GENERAL PRINCIPLE: INVOLVE
OTHERS
What do you needs to know/do to fulfill your assessment plan?
Do you have all the necessary skills?
Who does, and can you get them to help you?

10. GAME-SPECIFIC CHALLENGES
The conundrum of “fun”
Asynchronous games
Complexity

11. INTEGRATING ASSESSMENT INTO
GAMEPLAY
Can interactions include assessment components?
Can players complete assessment as part of gameplay?
What information can be collected unobtrusively? By staff? By
the system?

12. ELECTRONIC GAMES AND
METRICS
Number of players or interactions
Length of interaction
Path through the game or app
Privacy concerns

13. CASE STUDY: LIBRARYQUEST

14. LIBRARYQUEST
Asynchronous mobile game
Offers players “quests” or tasks
Complete for points, which can
be turned in for prizes
Learn by doing: accomplishing
quest = successfully using skill
Second pilot:Assessment focused

15. OBJECTIVES:
To teach incoming freshmen ten distinct skills and knowledge-
based goals relating to library services and spaces (example: how
to use the online catalog to locate a book).
To teach freshman that the library is a support service that exists
to help them in their work.
To increase freshman sense of self-sufficiency and confidence in
using the library.

16. INSTITUTIONAL AND NATIONAL
CONTEXT
ACRL Information Literacy Standards: PK Morgan Critique
“Pausing at the Threshold.”
Institutional IR assessment
Research on Student Retention
Emphasis on incoming freshmen

17. OUTSIDE COLLABORATORS
Head of Instructional Services
First-Year Experience Librarian
Statistical Consulting Center

18. ASSESSMENT COMPONENT: LEARNING
GOALS/STUDENT CONFIDENCE
Student selects quest
Quest app assesses student ability through skills-based quiz
Exposes results to student
Quest asks student to rate confidence on a ten-point scale
Student performs quest
Quest app asks student to rate confidence once again using ten point scale
App rewards student

19. EXAMPLE QUESTIONS: CALL
NUMBER QUEST
Which of the following is shelved first?
EA 301.E799 1990
E 302.E799 1989
E 302.A799 1999
Which part of this call number designates a general subject area?
HK
302.1
.E99
1990
On a scale of 0-9, rate how confident you are that you could find a book on the shelf using a call number:
Not Confident 0 1 2 3 4 5 6 7 8 9Very Confident

20. ASSESSMENT: IDENTIFYING
LIBRARY AS SUPPORT SERVICE
Pre and post-play questions:
Which of the following offer services that support you in completing your
assignments?
Registrar’s office
Writing Center
Student life office
University Libraries

21. EVALUATIVE DATA
Statistics:
Number of players
Number of quests completed
Which quests completed and abandoned
What steps completed before abandoned
When completed
What rewards were selected (and how many)

22. EVALUATION COMPONENTS
Periodic surveys throughout the game on elements of gameplay:
“What is your favorite quest so far? Why?”
Post-game Interviews
“If you could tell the designer of this game any one thing, what
would it be?”

23. Questions?