The Value & Economic Measures of Libraries

The Value & Economic
Measures of Libraries
10th Northumbria International
Library Conference
Joe Matthews
July 25, 2013
Morning Session

Outline
• Performance Measures
• Value
• Value of Information
• Value of Information
Services

Outline
Personal value
• Direct measures
• Indirect measures
Organizational value
• Academic libraries
• Public libraries
• National libraries
)

Few libraries exist in a vacuum,
accountable only to themselves. There is
thus always a larger context for
accessing library quality, that is, what and
how well does the library contribute to
achieving the overall goals of the parent
constituencies?
Sarah Pritchard

There is no systematic evidence
collected which shows the value
of academic libraries
for teaching and research staff.
Claire Creaser and Valerie Spezi

Performance Measures
Input Proces
s
Output Outcome
s
Outcome
s
Library
Services
Individual Society
Efficiency Effectivenes
s
Cost
Effectiveness
Impact
VALUE
Cost benefit
Resource
s
Capability Use Beneficial effects

Start with the end in mind:
work backwards
Refocus from the activity
to the impact

Library Control
How much? How many? How economical? How prompt?
Magnitude
% of change last
year
% of overall change
Cost
Magnitude
Change
Resources used
Units processed
Cycle times
Turnaround time
Anticipatory

Library & Customers Decide
How valuable? How reliable? How accurate?
Effort expended
Cost
Benefits obtained
Dependability
Access
Accuracy
Completeness
Comprehensiveness
Currency

Customers Decide
How well? How courteous? How responsive? How satisfied?
Accuracy
Promptness
Courtesy
Expertise
Attentive
Welcoming
Anticipatory
Helpful
Empathetic
Expectations met
Materials obtained
Personal interaction
Ease of use
Environment
Comfort
Willingness to
return

Challenges
Lack of consensus about what should be
measured and how
Lack of understanding of performance
measurement and metrics
Organizational structural issues
Lack of precision in measuring performance,
and
alignment issues
Determining the “bottom line” is too far away
Majority of stakeholders are too far away
Library staff find it difficult to see the “big”

Lack of a Connection
• Budget and outputs (and outcomes) are
separated
• No “bottom line” measure for libraries
• Decision-making process is bigger than
the library
• Library has neither champions nor foes
• Library benefits are not widely self-evident

Orr’s Fundamental Questions
• How good is the library?
• What good does the library do?
• How well is the library managed?

We should be a bit wary of the “little library”
…For when it is good, it is very, very good
and when it is bad,
it’s a “pretty good library for a town this size.”
Eleanor Jo Rodger

Levels of Assessment …
• Individual student
• Course
• Departmental/Program
• College or University

Types of Measures
• Direct
– Provide tangible, visible and
self-explanatory evidence of
what students have & have not
learned
• Indirect
– Capture students’ perceptions of
their knowledge & skills;
supplement direct measures;
sometimes called surrogates

Qualitative Tools
• Focus groups – open
ended
• Biography
• Phenomenology –
capture the “Aha!”
moment
• Grounded theory
• Ethnography
• Case study

Qualitative Assessment
• Provides in-depth understanding of user
responses and interactions
• Represents part of a long-term strategy of
formative evaluative

Quantitative Tools
• Surveys
• Transaction logs
• Statistics from systems
• Observations (count)

Quantitative Assessment
• Analyses to determine library impacts on
academic performance, retention rates
• Describe retention rates and GPAs in
defined populations over semesters and
users
• Compare users & non-users of library
services while adjusting for academic
preparation and background differences
• Conduct quasi-experimental designs
employing multivariate analysis of
covariance & hierarchical regression

Be cautious about cause-and-effect relationships

The Issue
• Is it: Use library resources/services and
you will get better grades.
• Or: I want to do well and so I work hard to
achieve better grades (and one way I do
that is to use library resources/services).

“Not surprisingly, librarians are keen to show
that the use of expensive, scholarly
materials
positively correlates with higher grades,
although they cannot prove that this is so.”
Deborah Goodall & David Pattern

“There is growing pressure on all academic
library managers to be more accountable for
how they use limited resources and to
achieve institutional outcomes perceived as
important by college and university
stakeholders….”
Elizabeth Mezick

Value of Information
• Expect value-in-use
• Library’s collection reflects a “potential
value”
• Collection also reflects a “future value”
• Value of local collection is declining

Valuable is not about our professional
values;
in the paradigm of the value of libraries,
we are the producers,
not the consumers of services.
Our sense of what is valuable
really doesn’t matter
much at all unless it
matches that our our customers.
Eleanor Jo Rodger

Fundamental Changes
Libraries have changed more in the past two
decades than in the prior two centuries.
Technologyis the major driver . . .
We need to recognize that all this change
has only begun, and that change is
irreversible.

Increasingly it is important
to remember that libraries
provide few unique services.

Information is woven into our
lives

Quality of Information
This fast food approach to information consumption drives
librarians crazy. “Our information is healthier and tastes
better
too” they shout. But nobody listens. We’re too busy
Googling.”
Peter Morville

Key Characteristics of
Information
Uncertainty Knowledge
Ambiguity Indeterminacy
Redundancy System dependency
Sharing Timeliness
Compression Presentation
Stability Multiple life cycles
Leakability Substitutability

Criteria for Judging Value
Customer Criteria Value Added by the Service
Ease of use Browsing, formatting, mediation service,
orientation service, ordering, physical
accessibility
Noise reduction Access (item identification, subject
description, subject summary), linkage,
precision, selectivity
Quality Accuracy, comprehensiveness, currency,
reliability, validity
Adaptability Closeness to problem, flexibility,
simplicity, stimulatory
Time savings Response speed
Cost savings Cost savings

Collections are disrupted
Atoms to bits

Nature of Information is
Changing
Scare, controlled
Expensive
Shaped by elites
One-way, mass
consumption
Slow moving
External to our
worlds
All around us
Cheap or free
Shaped by consumers
Designed for sharing,
participation &
feedback
Immediate
Embedded in our
worlds
Information
was ….
Information
is ….

Ifthe physical proximity of
print collections had a demonstrable
impact on researcher productivity,
no university would hesitate to
allocate prime real estate
to library stacks.

Traditional Value Proposition
Without a great
library, there can
be no great
university.
David Kinly,
President of
The University of Illiniois
1929

Universities Provide
• Private goods &
services
– Courses exchanged for
tuition
– Research completed for
funding
• The value proposition
The value to an individual or an
organization determines

Academic Libraries Provide
• Public goods and
services
Print and online resources are
shared by all, usually without
the exchange of payment
• Value proposition
The collective value of all users
must be estimated to determine
if a good or service should be

Value is determined by the user
and the
useof information

Astin’s IEO Model
Input Output
Environment
Programs
Institutional
characteristic
s
Library
Fellow students
Faculty
Place of residence

Student Learning Outcomes
Model
Intelligence
General Reasoning
Broad Abilities
Knowledge, Understanding, and
Reasoning
Abstract,
Process
Oriented
Concrete,
Content-
Oriented

Campus Culture of Learning
• What learning experiences
require use of library
resources?
• What % of students engage
in learning experiences in
the library?
• Does the library contribute
to civic learning
expectations?
• Library $ spent that
support teaching and
learning?
• Consider using flourishing
and resiliency scales to
help identify the library’s
impact.
• What % of librarians serve
as student advisors?

Define, develop, and measure
outcomes
that contribute to
institutional effectiveness
ACRL Standards for Libraries in Higher Education

Challenge
It is not how much a library
is used that matters,
rather how does the library
impact or benefit the
customer?

Perspectives on Value
Benefits
Use
Nonuse
Direct
Indirect
Option – Preservation of option for
future use by me
Existence – Perceived value and
significance
to the community
Legacy – Value of preservation for
future generations
Personal
Organizational
Financial
Impacts

Why Use the Library?
Reasons Interactions Results
For a TASK
For PERSONAL
reasons
To get an
OBJECT or
INFORMATION
To perform an
ACTIVITY
Access
RESOURCES
Use of
RESOURCES or
SERVICES
OPERATIONS
ENVIRONMENT
S
COGNITIVE results
AFFECTIVE results
ACCOMPLISHMENTS
EXPECTATIONS met
TIME aspects
MONEY estimates

Generic Learning Outcomes
Knowledge & Understanding Skills
Attitudes Enjoyment, Inspiration, Creativity
Activity, Behavior, Progression

• Full-time students
• Live on campus
• Interact more with faculty
• Study more
• Collaborate with their
peers
Student Learning is Affected by
…

NSSE
• 5 benchmarks of effective educational
practice
– Level of academic challenge
– Active & collaborative learning
– Student-Faculty interaction
– Enriching educational experiences
– Supportive campus environment
• Student self-reported gains in intellectual &
personal development
• No overlap between self-reported data &
standardized objective tests

NSSE – Use of the Academic
Library
• 50% never used the library
• Use of libraries at small, academically
challenging liberal arts colleges are
correlated with other purposeful activities
• Library use less intensive at larger
universities
• Students who work harder use library

Assessing Student Achievement
• Direct measures
– Capstone experience
– Use of a portfolio
– Standardized exam (Collegiate
Learning Assessment)
• Indirect measures

• Gains in student
performance are
quite low
• Individual learning
is characterized
by persistence
• Notable variation
within and across
institutions
Assessment of Higher Ed

Wabash National Study
• Different instrument – CAAP
• 2,212 students
• Nearly identical results to Academically
Adrift
• 44 percent no gains in the first year
• 33 percent no gains in 4 years
• Students only study about 15 hours per
week

“One way to demonstrate the library’s
contribution is to assess whether students’
experiences with the library
directly or indirectly
contribute to desired outcomes of college.”
George D. Kuh & Robert M. Gonyea

How to Demonstrate Impact in
…
• Student enrollment
• Student experiences
• Student learning
• Student grades (GPA) &
achievement
• Student retention & graduation
• Student career success
• Faculty productivity
• Institutional reputation
• The environment

• Recruitment of prospective
students
• Matriculation of admitted students
• Recommendation of current
students
Student Enrollment

• GPA
• Professional/educational test
scores
• Learning assessments
• Faculty judgments
Student Learning

Meta-analysis
• Entering student characteristics
– SES
– High school GPA
– ACT/SAT scores
• Environment - Psychosocial and study
skill factors
– Academic goals, skills and self-confidence
– Social support & engagement (acculturation)

Student Learning Occurs …
• in the classroom
• in the laboratory
• with peers
• in the student union
• in the dorm
• in the library (for some)
• online
• and

Direct Measures
• Capstone Experience
• Portfolios
• Standardized Test
• Locally-developed
test

Indirect Measures
• Surveys
• Retention, graduation rates
• Grades
• Acceptance rates into
graduate programs
• Job placement rates
• Exit interviews
• Alumni surveys

Usage of Electronic Resources
WAM = Weighted Average Marks (Grades)

Other Studies
• Hong Kong Baptist University
• Georgia State University

University of Minnesota
Gym Bags and Mortarboards
Use Campus Recreational Facilities
At least 25 times, first-
year retention increased
1%
&
5-year graduation rates
increased 2%

University of Minnesota Library
• 5,368 first-year non-transfer students
• Use of library was associated with a .23
increase in students GPA
• More use of the library, GPA also goes up

University of Minnesota Library

• Opinion surveys
• Skills testing
• Observed behaviors

Library Instruction and GPA
University of Wyoming Libraries
• Analysis of 4,489 transcripts
• Slight positive relationship between upper-
level library instruction courses and GPA –
0.075 GPA difference – that’s less than
1/10th of 1 percent

• Research statement – 44%
• Evaluate Web site – objectivity –
52%
– authority – 65%
• Presentation to persuade – 12%

Library Instruction and GPA
Hong King Baptist University
• 45 sample groups – N=31 to 1,223, study
majors
• Pairs of data
• One-fourth (11) had a positive relationship
• Results:
– 1 or 2 workshops – little impact on GPA
– 3 or 4 workshops – ½ show a positive impact
– 5 workshops (1 sample group) – 100% had a
higher GPA

Bibliographic Instruction
• Improvement in basic library skills is the
means and not the end
• Yet the means is the focus for evaluation
efforts
• Evaluation efforts focus on
– Opinion surveys
– Skills improvement
– Pre-test & post-test knowledge
• Not the impact on student achievement

Library experiences do not seem to
directly
contribute to gains in information
literacy,
to what students gain overall in college,
or
to student satisfaction.
Kuh & Gonyea

How scalable is library instruction?

Student Retention & Graduation
Attrition
Retention
Persistence
Completion
Graduation
Rates

• Fall-to-fall retention
• Graduation rates
Student Retention

Retention Concepts
• Institutional retention
– Enrolling & graduating from the same
institution
• Program retention
– Enrolling & graduating with the same
major/department/school
• System retention
– Students who leave one university yet
continue and complete post-secondary
studies elsewhere

Measures of Retention
• Persistence (Continuation rate)
– From first to second year? Entry to
graduation?
• Completion rate
– From entry to graduation (Student goals?)
• Graduation rates
– Are transfers included? Time period?
• Attrition
– Leaving university? Leaving higher ed?

Measures of Retention
• Stopout
– Leave university with the intention (and
action) of returning later to complete a
program
• Dropout
– Leave university with intention (and action) of
NOT returning
• Transfer
– Change institutions yet persist in higher
education
– May change type of institution

Why Students Leave?
• Students’ decision to leave University is influenced
by many personal factors
– Financial reasons
– Family responsibilities
– Lack of academic ability
– Poor fit, etc.
• Foundational Theories from Education / Psychology:
– Tinto’s “Model of Student Integration”
– Bean’s “Model of Student Attrition”

Tinto’s
Model of Student Integration
Pre-entry
Attributes
Goals /
Commitment
s
Institutional
Experience
s
Integration Goals /
Commitment
s
Outcome
SES
Skills &
Abilities
Quality
of
Educatio
n
Intentions
Institutional
Commitment
s
Academic System
Performance
Engagement
Extra-
Curricular
Peer group
Social System
Academic
Integration
Social
Integration
Intentions
Goal
Institutional
Match
Stay
or
Leave

Bean’s
Model of Student Attrition
Grades
Courses
Educational Goals
Major & Job Certainty
Opportunity to Transfer
Family Approval
Organization
alVariables
Personal
Variables
Environment
al
Variables
Loyalty Attitudes
Certainty
Practical
Value
Intent
Dropout

Student Retention & Graduation
• Important because … rankings,
revenues, educational achievement,
emotional well-being
• Many reasons for drop-outs are not
under the control of the university
• Engagement is the key

Indicators
• Student goal
attainment
• Course retention
• Subsequent course
work
• Fall-to-fall persistence
• Time to degree
• Degree completion
• Grad school
enrollment
• Transfer rate & success
• Employer assessment
• Academic value add
• Student satisfaction
• Professional growth
• Student involvement
• Citizenship &
engagement

Australasian Survey of Student
Engagement (AUSSE)

“A high rate of attrition
is indicative of a failure
on the part of an institution
to achieve its purpose.”
Elizabeth Mezick

Student Engagement
“many students
don’t develop a
personal connection
with their institution.
And when they
don’t, they leave”
Gonzales 2010 NSSE &
CSEQ

Library Retention Studies
• Statistically significant relationships between
library expenditures, or staffing levels and
student retention
 E.g. Hiscock, 1986
Hamrick, Schuh, & Shelley, 2004
Mezick, 2007

Graduation Rates & Library
Expenditures
• Used IPEDS data on institutional
characteristics & resource allocations
• Library expenditures was strongly
correlated with graduation rates – 1.77
percent increase in graduation rates
• Greatest payoff is attributable to enhanced
library expenditures (+0.92) and instruction
(+0.80) while increased non-library
contributions were quite modest (+0.27)
Hamrick, Schuh & Shelley

• Relationships between library use
(collections) and student retention
– Student who borrowed books = more likely to
persist
• E.g. Kramer & Kramer, 1968
• Impact of instruction
– Students involved in library skills programs
showed lower attrition rates
• E.g. Knapp, 1966

University of Minnesota
– 77% of undergrads made use of the
libraries, 85% of grad students made use of
the libraries
– Students who used the library at least once
were 1.54 times more likely to re-enroll

• Some library involvement in first year
experience programs; specific programs for
“at risk” groups
– NOT proven to have significant effect
• E.g. Hollis, 2001
Colton, et al, 2002
Aguilar & Keating, 2009
Love, 2009

• Relationship between library employment
& retention
– Higher completion rate among library student
workers
• E.g. Wilder, 1990
Rushing & Poole, 2002

“If strong linkages between libraries
and student retention can be made,
then the perceived value of the library
may indeed rise.”
Steven Bell

• Some groups, some majors & seniors
engage in more library-related activities
• Academic support expenditures tend to
correlate with increased engagement
• Institutional academic challenge
correlates with library use

Student Career Success
Grad School Exams
Alumni Surveys

• Job placement rates
• First-year job salaries
• Professional/graduate school
acceptance
• Internship success
• Marketable skills
Student Success

• Integration of library resources and
services into course syllabi, Websites,
lectures, labs, reserve readings, etc.
• Faculty/librarian collaborations;
cooperative curriculum, assignment, or
assessment design
Faculty Teaching

Perceived Benefits for Teaching
• Savings
– Of own time
– Of own money
– Of other resources
• Improvements
– Teaching
– Course-related materials
– Student performance

Impact on Faculty
• Library is the source for most journal
articles (individual subscriptions are way
down)
• If library subscriptions were unavailable –
productivity would decrease 17%
• Library is not the source of book readings
• 42% of reading material is library provided

Time
• Academics spend a lot of time reading
• Article reading inspires new thinking,
improved results, changed focus
• Award-winning academics read more
• Academics who publish more use more
library resources

Ithaka Studies
• Library services not
understood
• Library services not valued
• The Library is
disappearing

• Number of publications, number of
patents, value of technology transfer
• Tenure/promotion judgments
Faculty Research Productivity
Faculty Grants
• Number of grant proposals (funded or
unfunded)
• Value of grants funded

Assessment of Research
• Payback model – form of ROI
• Research impact
• Research utilization ladder
• Lavis decision-making impact model
• Weiss logic model
• HTA organization assessment
framework
• Societal impact framework
• Research assessment exercise
• Becker medical library model

For Most Impact Models
• Indicators of research output
• Indicators of knowledge transfer
• Indicators of implementation
• Indicators of community benefit

• Faculty recruitment
• Institutional rankings
• Community engagement
Institutional Reputation & Prestige

Institutional Reputation
• Changes in reputational rankings affects
student & faculty recruitment
• University budget allocations to libraries
have decreased
Since the library absorbs a very small percentage of a
university budget, the contribution of the library is
disproportionately high relative to its cost to the institution.
Sharon Weiner

• Indispensable for their research
• Maintain a high-level overview of their field
• Value for money is good
• Library not available, costs would increase
40%
• Take 31% longer to locate same information

Library Value Scorecard
• Relational Capital
• Library Capital
• Library Virtue
• Library Momentum

University & the Library Can
• Attract outstanding faculty
• Retain outstanding faculty
• Foster innovative research
• Align library activities with
university goals

Value of Special Libraries
• Time saved
• Money saved
• New revenues
• Other outcomes

In addition, to ROI
• Knowledge-Value Add
• Intranet Team Forums
• Intellectual Capital
Valuation

Measurements of value
were, in fact, a
key differentiator
between
successful
and unsuccessful corporate libraries
James Matarazzo

Social Capital
• Bonding social capital
• Bridging social capital
• Linking social capital

Social Benefits
• Basic reading literacy
• Business/career
• Information literacy
• Library as place
• Summer reading
• Local history &
genealogy
• Health & well-being
• Social cohesion
• General information
• Empowerment

• Social connection
• Education
• Employment
• Health & wellness
• eGovernment
• Community engagement
• Personal finance

Making Cities Stronger
• Building a stronger local economy
• Improving early literacy & school
readiness
• Building workforce participation
• Supporting small business
• The power of place

• Literacy
• Workforce development
• Business development
• Value to homes and
neighborhoods

Libraries/Building/Communities
• Developing social capital
– Providing a welcoming environment
– Creating a pride of place
– Attracting users from all walks of life
– Reaching out to the community
– Appreciation of cultural differences
– Building bridges to government
– Encouraging collaboration across the
community

• Overcoming the digital divide
– Making technology accessible
– Exploiting technology to benefit the
community
• Creating informed communities
– Community information
– Government information
– Providing a gateway to the world of
information

• Convenient and comfortable places of
learning
– Developing information skills
– Stimulating ideas and discussion
– Supporting vulnerable learners
– Supporting students

Outcomes
Enhanced quality of life
Enhanced enjoyment from
hobbies
Able to obtain information
Facilitates lifelong learning
Support for children’s education

Contributions
Safe and pleasant place
Supporting educational facilities
Facilitating lifelong learning
Encouraging responsible social
behavior
Ensuring access to the Internet

Tracking Value
The Engaged Library:
Chicago Stories of Community Building
•Prove that public libraries build social capital
•Identify & connect the library’s assets to the
community
•Assess & strengthen the library’s connections with
and use of community assets
•Produce a toolkit for other libraries to adopt to
•Mapping tools to perform an inventory services,
identify areas for improvement and highlight library’s
contribution to the community’s wider social,
educational, cultural and economic goals.

PLQIM
• Access to information
• Community & personal participation
• Meeting readers’ needs
• Learners’ experiences
• Ethos & values
• Organization & use of resources
• Leadership

Valuing the Collection
Dewey Subclass
Number of Titles
2010-2011 Avg
List Total Value ($)
001 - Knowledge $76.71 $0.00
002 - The book $62.45 $0.00
003 - Systems $129.77 $0.00
004 - Data processing. Computer science $89.82 $0.00
005 - Computer programming, programs, data $69.14 $0.00
006 - Special computer methods $83.60 $0.00
010 - Bibliography $73.65 $0.00
011 - Bibliographies $69.46 $0.00
012 - Bibliographies of individuals $0.00 $0.00
013 - Of works by specific classes of authors $0.00 $0.00
014 - Of anonymous and pseudonymous works $55.95 $0.00
015 - Of works from specific places $184.99 $0.00
016 - Of works on specific subjects $134.87 $0.00
017 - General subject catalogs $0.00 $0.00
018 - Catalogs arranged by author, date, etc. $0.00 $0.00
019 - Dictionary catalogs $0.00 $0.00
020 - Library and information sciences $56.06 $0.00
021 - Library relationships $62.60 $0.00
022 - Administration of the physical plant $65.50 $0.00
023 - Personnel administration $56.00 $0.00
025 - Library operations $77.02 $0.00
026 - Libraries for specific subjects $89.99 $0.00
027 - General libraries $66.25 $0.00

Joe@JoeMatthews.Org
www.joematthews.org
Joe Matthews
Library Consultant

What Are the Results
Four Year
College
Full-time
Four Year
College
Part-time
Two Year College
Stopout/Transfer
College degree – 19%
AA degree – 8%
No college degree –
60%
No college
Graduate degree –
11%

The Value & Economic Measures of Libraries

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (17)

Destaque

Destaque (15)

Semelhante a The Value & Economic Measures of Libraries

Semelhante a The Value & Economic Measures of Libraries (20)

Mais de Joe Matthews

Mais de Joe Matthews (19)

Último

Último (20)

The Value & Economic Measures of Libraries

Notas do Editor