Libraries collect and use many different types of statistics, but effectively managing them is a challenging opportunity for libraries to understand statistical trends through analysis. Stephanie H. Wical and Hans Kishel surveyed and interviewed Wisconsin academic libraries in order to understand what statistics these libraries currently collect. What tools and measures do Wisconsin academic libraries use and for what purpose? What do these libraries consider best practices? New tools have been developed that help manage statistics but questions remain. How do we compare statistics across vendors? What measures are libraries using and why? Have Wisconsin academic libraries discovered effective ways to manipulate and present their data? How do libraries value and prioritize uses of statistics that they collect? Is it possible to analyze statistics to paint a compelling picture to justify collection development decisions or planning? Lastly, how can we begin to assess how successfully we use statistics for various purposes? This presentation explores these questions as well as several case studies that highlight examples of real libraries and how they manage statistics. It is our hope that this talk will help engage library staff in a discussion about how to most effectively manage library resources. Presented by Stephanie H. Wical on behalf of Stephanie H. Wical and Hans Kishel.

1. Strategic Collection
Management through
Statistical Analysis
Stephanie H. Wical
University of Wisconsin – Eau Claire
&
Hans Kishel
University of Wisconsin – Eau Claire

2. Methodology
• Identified academic libraries in Wisconsin using a
directory maintained by Wisconsin Association of
Academic Libraries.
• Surveyed academic librarians who were believed
to have some role (if not sole responsibility) for
electronic resource management.
• Interviewed academic librarians (who were a
subset of the group who responded to the
survey).
NASIG 2012

3. Demographics of the Group
The group included libraries in public and
private colleges and universities, technical
colleges, two-year colleges and for-profit
colleges and universities in the state of
Wisconsin or academic libraries that provide
library service to residents of the state of
Wisconsin.
NASIG 2012

4. Response Rate
• 139 surveys distributed via e-mail
• 62 completed survey responses
• 28 completed phone interviews
45% Response Rate for the Survey as of May 25, 2012
NASIG 2012

5. Survey Question: What types of library electronic
resources statistics--if any--do you currently collect?
(Choose all that apply.)
N=62
Answer Responses Percentage
Searches 54 87%
Sessions 46 74%
Full Text Downloads 41 66%
Cost-per-use 25 40%
IP Addresses that access resources 3 5%
Simultaneous Users 9 15%
Turn-aways 27 44%
Times authors at your college or university are cited in resources 1 2%
Times authors at your college or university cite resources 2 3%
Scholarly Impact 1 2%
Not sure 3 5%
None 1 2%
Other (Please specify.) 6 10%
NASIG 2012

6. Survey Question: What measure or measures do you
consider when evaluating electronic resources in your
library? (Choose all that apply.)
N=62
Answer Response Percentage
Searches 53 85%
Sessions 39 63%
Full Text Downloads 38 61%
Cost-per-use 37 60%
IP Addresses 1 2%
Simultaneous Users 19 31%
Turn-aways 21 34%
Times authors at your college or university are cited in resources 3 5%
Times authors at your college or university cite resources 3 5%
Scholarly Impact 20 32%
Not sure 2 3%
None 1 2%
Other (Please specify.) 7 11%
NASIG 2012

7. A Comparison of Collected and Considered
60
50
40
30
20
10
Considered
0
Collected
N=62
NASIG 2012

8. Searches, Sessions, Full Text Downloads and Turnaways
NASIG 2012

9. Survey Question: How frequently do you
consider your measure or measures?
Never
2%
Other (Please
specify.)
Not sure 16%
5%
Once a year
39%
Monthly
16%
During the
fall and
spring
semesters
Quarterly 16%
6%
N=62
NASIG 2012

10. Interview Question: Do you look at cost-per-
use for electronic resources?
No
32%
Yes
68%
N=28
NASIG 2012

11. IP Addresses That Access Resources
This measure might be important down the
road as departments outside of the library
might have to share costs for resources.
NASIG 2012

12. Times Cited or Times Citing
Collecting this type of information was not a high
priority for any of the librarians we talked to.
NASIG 2012

13. Scholarly Impact
• Scholarly impact is not something one routinely
tracks, but it is something that is valuable when
considering adding a new resource.
Image from Experimental Physiology at http://ep.physoc.org/site/misc/author.xhtml
NASIG 2012

14. Interview Question: What measure or measures do you
consider when evaluating electronic resources in your library?
N=28
70%
60%
50%
40%
30%
20%
10%
0%
NASIG 2012

15. Interview Question: Why do you use the measure or measures
you use to evaluate electronic resources?
30%
25%
20%
15%
10%
5%
0%
Budget We have always done To get idea of what Renewal To be able to compare
it that way / It is what students are using
we get
N=28
NASIG 2012

16. Interview Question: Do you feel that the measure or measures
you use are adequate or do you feel that they do not provide
relevant information?
N=28
No
21%
Yes
50%
So-So
29%
NASIG 2012

17. People who were not satisfied with usage
statistics indicated:
• Not all of them are counter compliant.
• Could always use more data.
• Sometimes usage statistics do not account for a lot
of variables.
• It would be nice to find a measure that can be looked
at across vendors.
• They are not relevant at all; statistics are a joke.
• We are drowning in data.
NASIG 2012

18. Where one library may see value in a set of
statistics, another library may need something
more.
---Schufreider & Romaine (2008)
NASIG 2012

19. Survey Question: What tool or tools do you use to gather
statistics for evaluating electronic journals, databases and/or
electronic books? (Choose all that apply.)
N=61
Answers Number Using Tool Percentages
SFX Ustat 5 8%
ERMes (developed at UW- La Crosse) 8 13%
Scholarly Stats 1 2%
Serials Solution 360 Link 4 7%
Serials Solutions 360 Resource Manager 5 8%
EBSCO ERM Essentials 4 7%
EBSCO LinkSource 7 11%
Ex Libris’ Verde 1 2%
Innovative ERM 1 2%
Open ERM 1 2%
OCLC Worldcat Link Manager 2 3%
CORAL ERM 1 2%
Excel spreadsheet or other spreadsheet software 40 66%
Access database or other database software 10 16%
Not sure 10 16%
None 2 3%
Other (Please specify.) 15 25%
NASIG 2012

20. Finding
66% are using Excel in addition to another tool
N=61
Serials Solutions 360 Resource Manager
ERMes (developed at UW- La Crosse)
OCLC Worldcat Link Manager
Serials Solution 360 Link
Other (Please specify.)
EBSCO ERM Essentials
EBSCO LinkSource
Innovative ERM
Ex Libris’ Verde
Scholarly Stats
CORAL ERM
Q6 What tool or tools do you use to gather statistics
Open ERM
SFX Ustat
Not sure
for evaluating electronic journals, databases and/or
electronic books? Excel spreadsheet or other
spreadsheet software
Excel spreadsheet or other spreadsheet software 4 7 1 3 2 4 6 1 1 1 1 1 1 7
NASIG 2012

21. Survey Question: Are you able to compare statistics
across vendors and platforms?
Other
(Please
specify.)
13%
Not sure
16% Yes
60%
No
11%
N=62
NASIG 2012

22. Survey Question: Have you taken advantage of the
Standardized Usage Statistics Harvesting Initiative (SUSHI)
protocol to obtain usage reports?
Other (Please
specify.)
7% Yes
7%
Not sure
16%
Plan to do so in
the next two No
years 57%
13%
N=61
NASIG 2012

23. Survey Question: How important are usage statistics in the
decision to renew or cancel electronic resources?
Very unimportant
8% Unimportant
2%
Neither
unimportant nor
important
Very important (neutral)
35% 16%
Important
N=62 39%
NASIG 2012

24. Survey Question: Have you ever canceled an electronic
resource because it had low use?
N=62
Not sure
No 11%
8%
Yes
81%
NASIG 2012

25. “How important are usage statistics in the decision to
renew or cancel electronic resources?”
&
“Have you ever canceled an electronic resource
because it had low use?”
How important are usage statistics in the
decision to renew or cancel electronic Have you ever canceled an
resources? electronic resource Grand
because it had low use? Total
Not
Yes No sure
Very unimportant 3 2 5
Unimportant 1 1
Neither unimportant nor important (neutral) 6 3 1 10
Important 20 1 3 24
Very important 21 1 22
Grand Total 50 5 7 N=62
NASIG 2012

26. What do you collect and how important is it?
Q3 What types of library electronic resources statistics--if any--do you currently collect? (Choose all that apply.)
IP Addresses that access resources
Times authors at your college or
Times authors at your college or
university are cited in resources
university cite resources
Other (Please specify.)
Full Text Downloads
Simultaneous Users
Scholarly Impact
Cost-per-use
Turn-aways
Searches
Not sure
Sessions
None
Q14 How important are usage statistics in the
decision to renew or cancel electronic resources?
Very unimportant 4 4 4 4 4 1 1 1
Unimportant 1
Neither unimportant nor important (neutral) 8 7 6 2 1 3 1
Important 21 19 16 8 2 7 11 1 1 1 3
Very important 21 16 15 11 1 1 9 1 2
Grand Total 54 46 41 25 3 9 27 1 2 1 3 1 6
NASIG 2012

27. Using Statistics
What will you do with usage statistics?
– Make changes and improvements
– Request additional resources based upon
data
NASIG 2012

28. Interview Question: What do you think that you should be
doing with electronic resource usage statistics?
Better
organized / Marketing
SUSHI 11%
12% What is
getting used
Renewal/informed 12%
decisions
19% Bang for the Buck
15%
Communication
19%
More indepth
12% N=28
NASIG 2012

29. Survey Question: Do you plan to collect additional
statistics in the next two years?
Do you plan to
0% collect
additional
Not sure
statistics in the
11%
next two
Yes years?
37% 0%
Maybe; it will
depend on
circumstances
(e.g. budget or
staffing)
42% No
10% N=62
NASIG 2012

30. Sharing Usage Statistics
• Who needs to know about usage
statistics?
– Identify stakeholders
– What will the reports look like?
– What is the right amount of information?
– Should the reports look different for
different people?
– Should an executive summary be written
for an administrator?
NASIG 2012

31. Survey Question: Are usage statistics for library
electronic resources reported outside the library?
Not sure
26%
Yes
50%
No
24%
N=62
NASIG 2012

32. Survey Question: Who receives regular electronic resources
statistical reports from the person or people who collect them?
(Choose all that apply.)
N=62
Answers Responses Percentages
Electronic Resources Librarian 10 16%
Periodicals Librarian 7 11%
Technical Services Librarian 5 8%
Director / Dean 30 48%
Administrator who supervises the director or dean of the library 7 11%
Reference Librarian 10 16%
Systems Librarian 5 8%
Everyone in the library 13 21%
Everyone on the e-resources team, work group or task force 4 6%
Anyone who reads the annual report 5 8%
Anyone who reads the library newsletter 0 0%
Anyone who reads the library blog, Facebook page, or twitter feed 0 0%
No one 7 11%
Not sure 4 6%
Other (Please specify.) 16 26%
NASIG 2012

33. Survey Question: Who collects electronic resources statistical
reports at your library? (Choose all that apply.)
N=62
Answers Responses Percentages
Electronic Resources Librarian 17 27%
Periodicals Librarian 7 11%
Technical Services Librarian 6 10%
Director / Dean 13 21%
Reference Librarian 6 10%
Systems Librarian 8 13%
Paraprofessional 13 21%
Student Workers 2 3%
We don’t collect them at our library---they are collected on a system or consortial
level 10 16%
Our vendor collects them for us 11 18%
We do not collect them at all 1 2%
Not sure 0 0%
Other (Please specify.) 10 16%
NASIG 2012

34. Survey Question: Is collecting electronic resources
statistical data something that is written into one or
more job descriptions?
Other
N=62 (Please
specify.)
2%
Not sure
18% Yes
37%
No
43%
NASIG 2012

35. Interview Question: What would you consider a best practice
for collection management statistics for electronic resources?
Answer # of responses %
No Idea 7 25%
ERM 2 7%
SUSHI 5 18%
Collect Regularly 1 4%
Check them to make sure they are right 1 4%
Big Picture thinking 2 7%
Compare over time 3 11%
Feedback 2 7%
Review Regularly 1 4%
How students use 1 4%
Cost per use 1 4%
COUNTER 2 7%
Use them 2 7%
N=28
NASIG 2012

36. Interview Question: How do you prioritize uses of the electronic
resources usage statistics you collect? Or what is most important?
N=28
Answers Number of Responses Percentages
What getting used 9 32%
Cost per use 1 4%
Renewal 6 21%
Annual Report 5 18%
Turnaways 1 4%
FT-DL 4 14%
Search 5 18%
Course integration 2 7%
Sessions 1 4%
Budget 4 14%
Faculty Requests 1 4%
Vendor Data 2 7%
ACRL Reports 1 4%
Don't 1 4%
Advertising 1 4%
Bang for Buck 2 7%
Look for Changes/Problems 2 7%
NASIG 2012

37. Assessment Process
NASIG 2012

38. Survey Question: Has there been any attempt at your library
to incorporate usage data into assessment of library services?
Not sure
16%
Yes
49%
No
35%
N=62
NASIG 2012

39. Interview Question: How do you determine if usage statistics
are evaluated and consulted successfully?
N=28
Feedback
26%
Don't
If there is a 59%
problem then
something is
wrong
15%
NASIG 2012

40. Statistics can play a more progressively
important role in determining the return on
investment with library dollars.
---Beals & Lesher (2010)
NASIG 2012

41. 80/20 Distribution
Thomas Nisonger (2008) states, “The basic 80/20
pattern provides a valid approach to
operationalizing the core journal concept and is
applicable to collection management decision
making” (p. 78).
Although Nisonger (2008) admits that the
percentages do not match the 80/20 rule
exactly, should we make sure that the majority of
our budget is going into resources that get the
majority of use?
NASIG 2012

42. Quantitative Data
– Cost-per-use
– Number of disciplines
– Historical usage data (including five year trends)
– Interlibrary loan data
– Impact factors
Source: Hulbert, Roach & Julian (2011)
NASIG 2012

43. Qualitative Data
– Experience or the gut feeling
– User feedback
– Reviews
– The dust factor for print resources
Source: Hulbert, Roach & Julian (2011)
NASIG 2012

44. Decisions and planning up front are often the
most time-consuming part of the process.
--Hulbert, Roach & Julian, 2011
NASIG 2012

45. Case Study I
Nancy Beals of Wayne State University created a “Stats Master” spreadsheet
and training program for selectors (Beals & Romaine, 2010).
NASIG 2012

46. Case Study II
Linda Hulbert & Dani Roach of University of St. Thomas Libraries developed
an “Electronic Resources Review Checklist” and a way to “connect the silos”
to contribute to the university’s overall culture of assessment (Hulbert, Roach
& Julian, 2011)
NASIG 2012

47. Case Study III
Mary Ann Trail and Kerry Chang-FitzGibbon describe how the Library of the
Richard Stockton College of New Jersey addressed a $29,000 shortfall using
an overlap analysis report (Trail, Chang-FitzGibbon & Wishnetsky (2012).
NASIG 2012

48. Case Study IV
Bryan Vogh and Hans Kishel (2011)at the University of Wisconsin – Eau Claire charted
the workflow necessary for adding a new database for the McIntyre Library to best
utilize BP Logix.
NASIG 2012

49. Case Study V
Carol Tenopir (2010) looks at ROI of the value of
e-journals to grants income.
• “I could not submit as many grants. With grant funding at 4-
6% of submitted proposals I would not have achieved my
current funding level.” (U.S. University)
• “I guess on average the online access saves me more than 10
hours per week.” (Western European Research Institute)
(p. 44).
NASIG 2012

50. Case Study VI-?
What You or Your Colleagues are Doing
(Audience Provides At Least One Example.)
NASIG 2012

51. References
Bob Schufreider & Sion Romaine (2008): Making Sense of your Usage Statistics, The Serials Librarian, 54:3-4, 223-227, from
http://dx.doi.org/10.1080/03615260801974164
Bryan Vogh & Hans Kishel (2011): E-forms: Making Workflows Work, UMWUG Conference in Fargo, ND, October 17, 2011, from
http://people.uwec.edu/voghbs/UMWUG_E-forms_Handout.pdf
Carol Tenopir (2010): Measuring the Value of the Academic Library: Return on Investment and Other Value Measures, Serials Librarian, 58:1-
4, 39-48, from http://dx.doi.org/10.1080/03615261003623005
Linda Hulbert, Dani Roach & Gail Julian (recorder)(2011): Integrating Usage Statistics into Collection Development Decisions, The Serials
Librarian, 60:1-4, 158-163, from http://dx.doi.org/10.1080/0361526X.2011.556027
Mary Ann Trail, Kerry Chang-FitzGibbon & Susan Wishnetsky (recorder)(2012): Using Assessment to Make Difficult Choices in Cutting
Periodicals, The Serials Librarian, 62:1-4, 159-163, from http://dx.doi.org/10.1080/0361526X.2012.652931
Nancy Beals & Marcella Lesher (2010): Managing Electronic Resource Statistics, The Serials Librarian, 58:1-4, 219-223, from
http://dx.doi.org/10.1080/03615261003625844
Sarah Glasser & Michael A. Arthur (recorder) (2011): When Jobs Disappear: The Staffing Implications of the Elimination of Print Serials
Management Tasks, The Serials Librarian, 60:1-4, 109-113, from
http://dx.doi.org/10.1080/0361526X.2011.556447
Thomas E. Nisonger (2008): The “80/20 Rule” and Core Journals, The Serials Librarian, 55:1-2,62-84, from
http://dx.doi.org/10.1080/03615260801970774
NASIG 2012

52. Thank You!
Stephanie H. Wical
Assistant Professor
Periodicals and Electronic Resources Librarian
McIntyre Library
University of Wisconsin – Eau Claire
105 Garfield Avenue
Eau Claire, WI 54702
(715) 836-3508
wicalsh@uwec.edu
Hans F. Kishel
Assistant Professor
Research and Instruction Librarian for Science
McIntyre Library
University of Wisconsin – Eau Claire
105 Garfield Avenue
Eau Claire, WI 54702
(715) 836-2959
kishelhf@uwec.edu
NASIG 2012