Demonstrating the library's impact through assessment and evaluation

Do you know your
organization's
VMG?

Do you know your
library’s VMG?

Do you know the
role of your library
in your
organization? In the
society?

Do you know your
library’s purpose?

DEMONSTRATING THE
LIBRARY’S IMPACT
THROUGH ASSESSMENT AND
EVALUATION
Christine M. Abrigo
De La Salle University Libraries
Managing Today’s Learning Commons:
Re-skilling Seminar for information professionals
Philippine Association of Academic/Research Librarians, Inc.-
20-22 September 2016, Baguio City

 Explain the importance of purpose and
evidence
 Explain why evaluate
 Explain the measures for evaluation
 Explain about impact and impact
evaluation (looking beyond the
numbers)
 Workshop
OBJECTIVESOFTHIS
LECTURE

Importance of
Purpose and
Evidence

Purpose
 the reason why something is done or
used (Merriam-Webster, 2015)
 something you do to achieve something
 the intention
 answers the questions: “Why”, “What for”
and “Who for”?
Aim
 the main and overarching stated purposes
of the library service (Markless & Streatfield, 2013)
HAVINGAPURPOSE

Why is having a purpose important?
 Library’s purpose = borne out of your
organization’s VMG
 Creating a purpose sets the direction to
what you want to achieve and how you
will achieve it
HAVINGAPURPOSE

So what do we have in mind?
USERS
 We want to satisfy their needs.
PROGRAMS/PROJECTS
 We want to improve our services.
RESOURCES
 We want to provide the best/top of
the line.
HAVINGAPURPOSE

So what are we inclined to do?
 We come up with new services,
programs, projects, build resources
(aside from the traditional operational
services). Why?
 We come up with activities under these
programs and services
HAVINGAPURPOSE

BOTTOMLINE:
 Are these working based on your
purpose?
 Are these effective?
 Are we creating impact?
HAVINGAPURPOSE

We need evidence…
 to show that change has occurred
 to show effectiveness (that the
programs, services, projects that we do
is making a difference)
 to prove that what we are doing is
meaningful
 to demonstrate our value to our parent
organization, learners and community
THENEEDFOREVIDENCE

Assessment Evaluation
the systematic collection of data
to monitor the success of a
program or course in achieving
intended outcomes
the process of determining worth,
merit, or value of something;
assessment of “goodness”
judging people’s knowledge or
skills; getting a baseline of what
your stakeholders know and what
they need
a judgment about whether the
program has met its intended
outcomes
Mostly descriptive and
quantitative
Mostly qualitative

Why evaluate? Because…
 it tells you how well the library is performing
 a library rarely does it (comes up with a
service/program, implements, and leaves it
be)
 a library focuses on inputs and outputs (no
impact)
 a library would likely be inclined to calculate
efficiency (i.e., making the most out of
resources/service to produce a desired
result) using traditional performance
indicators (e.g., usage stats, entrants,
assessment studies, ROI, et al.), and not
effectiveness

Levels of analysis
 Individual (i.e., user experience)
 Service (i.e., user group’s experience)
 Organizational (i.e., all library users)
 Societal (i.e., impact on local
community)
MEASURESFOR
EVALUATION

Inputs
(resources; What
are needed)
Process
(activities; What
to do)
Outputs
(products;
What the
results are)
Outcomes
(effect/turnout/benefits;
What the consequences
are)
MEASURESFOR
EVALUATION Basic model
(Richard Orr, 1973)

MEASURESFOR
EVALUATION Inputs Process Outputs Outcome
 IL training
courses
 IL reference
materials
 Program
 Resource
persons
 Students
 Trainings/Work
shops for
librarians
 Development of
IL program
materials
 modules
 IL sessions
 quizzes and
games
 self-paced IL
learning tools
 instructional IL
videos and
guides
 information
products (e.g.,
pathfinders,
guides, etc.)
 Librarians know
what relevant
information are
for appropriate
users
 Users become
confident in
finding
authoritative
sources
 Users become
independent
learners and
practice
information
ethics
Sample: Information Literacy Program

On impact and
impact evaluation
(looking beyond the numbers)

Let’s review:
 You identify your VMG
 Create your library’s purpose aligned to your
organization’s VMG
 Demonstrate evidence of your value to your
community
 Come up with initiatives (services, programs),
noting your purpose and aims
 Evaluate these initiatives
Are you creating impact?
ONIMPACTANDIMPACT
EVALUATION

Impact is...
 seeing change or difference in what you
do
 concerned about effectiveness
 any effect (maybe positive or negative)
of the service on either an individual,
group or both
 long-term; requires time
 a series/assessment of outcomes
ONIMPACTANDIMPACT
EVALUATION

Measuring impact
Impact evaluation (outcomes-based evaluation)
 largely focused on monitoring service
efficiency
 systematic causation (cause-effect)
 requires clear and consistent use of
terms
ONIMPACTANDIMPACT
EVALUATION

Measuring impact
Impact evaluation (outcomes-based evaluation)
Seeks to answer:
 “How much better off are your patrons (in terms of
service) after introducing a service/program?”
 “Does the program/service/resource have a different
impact on different groups?”
 "Did the program/service/resource cause the impact?”
 “What would have happened if the
program/service/resource had not taken place?”
ONIMPACTANDIMPACT
EVALUATION

METHODSFORIMPACT
EVALUATION
QUANTITATIVE
measuring changes in competences or
behavior or to find correlations between
library use and a person’s academic or
professional success
QUALITATIVE
evaluating users’ experiences and
opinions
user skills tests before and after a
training or the use of a library
service
surveys (print, telephone, or
online)
performance monitoring/data
mining
Interviews
unobtrusive observation focus groups, discussion groups
analysis of citations in course work
or research publications
users’ self-assessment of skills and
competences gained
comparison of success data with
use data

METHODSFORIMPACT
EVALUATION
Inputs Process Outputs Outcome
Impact
(changes in
confidence,
competencies/skills,
behavior)
 IL training
courses
 IL reference
materials
 Program
 Resource
persons
 Students
 Trainings/Works
hops for
librarians
 Development of
IL program
materials
 modules
 IL sessions
 quizzes and
games
 self-paced IL
learning tools
 instructional IL
videos and
guides
 information
products (e.g.,
pathfinders,
guides, etc.)
 Librarians know
what relevant
information are
for appropriate
users
 Users become
confident in
finding
authoritative
sources
 Users become
independent
learners and
practice
information
ethics
• Librarians become
library research
consultants and
collaborators
• Users become
independent
learners and
manage
information use on
their own
• Users succeed in
certification exams
and in finding
employment
Sample: Information Literacy Program

IMPACTEVALUATION
PROCESS Step 1: Impact Objectives
Step 2: Impact Indicators
Step 3: Evidence to collect
Step 4:
Use the evidence to improve
services and secure service
sustainability

Focus on Step 1
Impact areas  Impact objectives
 Identify key areas in your service/program
that you want to make an impact (e.g., in
IL, finding information)
 Use specific terms
 Time-limited (e.g., achievable in 3 years)
IMPACTEVALUATION
PROCESS

Focus on Step 1
Impact areas  Impact objectives
Objectives are often confused with
activities.
 Objectives – what you want to achieve
 Activities – what you will do to make the
objectives happen
IMPACTEVALUATION
PROCESS

Quiz
Objective or Activity?
(Markless & Streatfield, 2013)
1. To support managers in carrying out their
work effectively and efficiently.
2. To provide ICT-based learning in IL
sessions
3. To enable students to operate effectively
in an e-environment
4. To provide materials appropriate to the
needs of graduate students
IMPACTEVALUATION
PROCESS

Quiz
1. To support managers in carrying out
their work effectively and efficiently.
2. To provide ICT-based learning in IL
sessions
3. To enable students to operate
effectively in an e-environment
4. To provide materials appropriate to
the needs of graduate students
IMPACTEVALUATION
PROCESS

Quiz
5. To increase the confidence and competence
of socially excluded people in using library
resources
6. To develop children as readers
7. To collaborate with faculty in order to
integrate the library into their curriculum
8. To develop the skills of the library staff in
delivering effective user-education
IMPACTEVALUATION
PROCESS

Quiz
5. To increase the confidence and
competence of socially excluded people
in using library resources
6. To develop children as readers
7. To collaborate with faculty in order to
integrate the library into their
curriculum
8. To develop the skills of the library staff in
delivering effective user-education
IMPACTEVALUATION
PROCESS

Before you even think
about coming out with a
program/project/service
for your users...

 Think about the purpose for your
initiatives first, and then make sure that it
supports your organization's VMG.
 Think about impact in planning your
initiatives. Always integrate evaluation.
 Be convinced that your library has value
to your user community and organization.
Think lifelong learning effect.

DEMONSTRATING THE LIBRARY’S IMPACT
THROUGH ASSESSMENT AND EVALUATION
21 September 2016
Christine M. Abrigo
christine.abrigo@dlsu.edu.ph

REFERENCES Bawden, D., Calvert, A., Robinson, L., Urquhart, C., Bray, C., & Amosford, J.
(2009). Understanding our value; assessing the nature of the impact of
library services. Library and Information Research, 33(105).
Blixrud, J. C. (n.d.). Evaluating library service quality: use of LibQUAL+.
Retrieved from http://libqual.org/documents/admin/blixrud.pdf
Broady-Preston, J., & Lobo, A. (n.d.). Measuring the quality, value and impact
of academic libraries: the role of external standards.
Markless, S., & Streatfield, D. (2013). Evaluating the impact of your library.
London: Facet Publishing.
Matthews, J. R. (2007). The evaluation and measurement of library services.
Westport, Connecticut: Libraries Unlimited.
Poll, R. (2012). Can we quantify the library’s influence? Creating an ISO
standard for impact assessment. Performance Measurement and Metrics,
13(2), 121130. http://doi.org/10.18352/lq.7746
Poll, R., & Payne, P. P. (2006). Impact measures for libraries and information
services. Library Hi Tech, 24(4), 547–562.
http://doi.org/10.1108/07378830610715419
Streatfield, D., & Markless, S. (2009). What is impact assessment and why is it
important? Performance Measurement and Metrics, 10(2), 134141.

Demonstrating the library's impact through assessment and evaluation

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (10)

Semelhante a Demonstrating the library's impact through assessment and evaluation

Semelhante a Demonstrating the library's impact through assessment and evaluation (20)

Mais de Philippine Association of Academic/Research Librarians

Mais de Philippine Association of Academic/Research Librarians (18)

Último

Último (20)

Demonstrating the library's impact through assessment and evaluation

Notas do Editor