This document summarizes research on faculty satisfaction with journal collections at ARL libraries from 2006-2009. The key findings are:
1) Faculty satisfaction with journal collections showed no significant change over the four year period. Dissatisfaction remained consistently high.
2) A strong correlation continued to exist between low satisfaction with journal collections and low overall library satisfaction.
3) Follow-up interviews at Columbia University found that faculty desires included complete online access and coverage, strong research support services, and improved search capabilities.
4) While collections are important, faculty emphasized the need for quick solutions, workarounds, and seamless access to resources through the library.
1. Still Bound for
Disappointment?
Another Look at Faculty
and Library Journal Collections
Library Assessment Conference 2010
Jennifer Rutner, Assessment & Marketing Librarian, Columbia University Libraries
Jim Self, Director, Management Information Services, University of Virginia Libraries
3. 2006 Research Questions
• Given the substantial investment in journals,
why are faculty consistently dissatisfied with
their library’s journal collections?
• What is the relationship between journal
collections and overall library satisfaction
among faculty?
• How should we address the dissatisfaction?
4. 2006 Findings
ARL libraries are not meeting faculty wants and
needs when it comes to journal collections.
There is a correlation between overall
satisfaction with library services and journal
collections.
http://www.arl.org/bm~doc/arl-br-257-bound.pdf
5. 2006 Findings
A variety of factors influenced faculty
perceptions at UVA.
• access is confusing
• foreign language coverage
• incomplete backfiles
• physical access
• remote access
• browsing facilities
7. 2009 Research Questions
• Are faculty at ARL libraries still dissatisfied with
journals?
• Is the correlation between journal collection
satisfaction and overall satisfaction still
significant?
• Are journal collections still the #1 issue for
faculty?
• Why are faculty perceptions negative at
Columbia?
9. Data
2006, 2007, 2008, 2009
LibQUAL+ Notebooks
for participating ARL
Libraries with >50
faculty respondents.
Methodology
Same as UVA in 2006.
Quantitative analysis of
LibQUAL+ faculty data
from ARL libraries.
Follow-up phone
interviews at Columbia.
18. Is there a difference in scores from year to year?
(ANOVA)
• 2006-2009 adequacy gaps from each ARL
institution.
• P-value = 0.119, which is not deemed
statistically significant.
Faculty were no more or less dissatisfied with
journal collections in 2009.
19. Journals and Overall Satisfaction
5
6
7
8
9
-1.2 -1 -0.8 -0.6 -0.4 -0.2 0
Figure 8: LibQUAL+ 2009, Correlation of Faculty Satisfaction with Journal Collections (IC-8) and Overall Library
Service, 21 Libraries
r =0.71
23. What do our faculty say?
Columbia
Discipline
Phone Interviews
Conducted
N for LibQUAL+
2009
2009 IC-8
Departmental
Mean Adequacy
Gap
Architecture 5 8 -1.375
Business 6 8 -0.125
Computer Science 4 3 -0.333
Engineering 4 12 -0.583
History 0 22 -0.318
Humanities 5 60 -0.379
Math 1 4 -0.750
24. What do our faculty say?
• Support
• Search and online access
• Collection gaps
• Coverage
• Work-around
• Quick list
• Resources
• Print vs. Electronic
25. What do our faculty say?
Support
“What would be great for faculty would be if
when things are not available, there was one
source in the library, extraordinarily skilled at
tracking down items. […] These people would
be specialists in working the electronic and
journal capabilities.”
26. What do our faculty say?
Search and Online Access
“I think just having free text search, like Google
book search, would be something that would
be very, very useful to have. I still feel like we
are living 20 years behind where the rest of
the world is in terms of being able to search
these databases and large collections of books
that we have.”
27. What do our faculty say?
Work-Arounds
“I just buy them individually from my research
funds, so it’s coming out of my research
money.”
28. What do our faculty say?
Quick List
“If I was to give a suggestion, maybe to have
discipline-specific pointers that could help
each discipline find things. […] It’s more of an
interface issue than a collections issue.”
29. What do our faculty say?
Resources
“The size of the collection is not as important as
getting the current collection working as
smooth as possible. Before, when we used to
go to the library, we got service.”
30. What do our faculty say?
Print vs. Electronic
“A few years ago, I wouldn’t have said that. But, I
guess things have changed.”
31. What do our faculty say?
Remote Access
(((crickets)))
First time presenting these findings – will be at LAC 2010
Jim Self, at UVA
2003-2006
Findings
UVA has not done LQ since 2006
Follow-up on this study here at CUL –what has changed?
In 2009 Columbia did LQ for the third time. It was our “make or break” year with the survey, and luckily we got an excellent response. In preparation, and in looking at the data, I remembered Jim’s analysis from 2006, and called him to see if I could get that expenditures chart. Then I asked if he had thought about updating the study… three years later. Things have changed! Budgets hadn’t yet taken the hit, interfaces had continued to improve, more universities were moving to e-only. In the midst of all of this rapid technological change, had faculty perceptions of journal collections improved?
What did we do?
Same methodologies – same exact interview questions for phone interviews. Notebooks contain only the means. We did not have individual cases to work with, all calculations are based on the the overall faculty means for a institution.
Let’s look at some data!
This is our survey item.
First, let’s look at Columbia – what do faculty think of our journal collections? We see strong dissatisfaction. Nothing surprising here, as we saw in Jim’s results in 2006. In-line with results at Columbia over the years.
Faculty scores
IC8 adequacy gap = -0.34 – still the largest gap for faculty.
To frame this topic… What’s intrigued me in this issue of journal collections is that it’s one of the items for which staff and user perspectives vary dramatically. If we look at library staff perceptions of journal collections, we’re pretty satisfied. We purchase the collections, we know what’s there. So, what’s happening between us buying it, and them using it?
Are faculty at ARL libraries still dissatisfied with journal collections?Faculty scores – pretty similar!
21 libraries included in our sample.
These are the means.
Arranged by expenditure: CUL is #1 on the chart.
Again, UVA has not participated in LibQual since 2006.
Looks like the minimums are rising a bit.
Desired scores are very stable – NOT rising. (We cannot say that expectations, ie. Desired scores, are constantly rising. The data don’t support that.)
Perceived is also rising. (Until 2009, but we had a smaller N)
Have not performed ANOVA’s on each score: min, des, per
But, are we getting closer to meeting the minimum? No. When analyzing the adequacy gap, there is NOT a statistically significant change – we’re still just are far from meeting the minimum perceptions as we were in 2006.
Have perceptions actually changed, in a statistically significant way? NO.
One-way ANOVA looking at adequacy gaps for IC8.
Null hypo: The means are not the same.
Accounts for years in sequence, and different samples that CAN have the same participant, but don’t have to.
P < 0.01 to reject the null hypothesis.
Does the correlation between satisfaction w journal collections and overall satisfaction still hold?
Correlation in 2009 is .71
Correlation in 2006 .80
SA3: How would you rate the overall quality of the service provided by the library?
A correlation is a single number that describes the degree of relationship between two variables. It’s not a causal relationship. It can be positive or negative, and it is between 0 and 1.
Do other IC items correlate more strongly with overall satisfaction?
Adequacy gap correlations for each IC item with overall satisfaction. Note: correlations are based on the mean scores of each institution, rather than the case-level data, which is not available.
24 follow-up interviews at Columbia, in total. Targeted the disciplines which reported negative adequacy gaps in LQ2009. N’s for some of these were small, but we felt this was the best way to approach it.
An expectation for a level of quality service online as well as in person.
Some of our automated systems came up here, with ILL and BD, in particular.
Faculty want to know who to ask with questions – who has the expertise?
And, some want specific training with e-resources.
Obviously, many of our search interfaces could use improvement, as could vendor side search interfaces. As we’re implementing new search tools, like MetaLib, Summon, CLIO Beta, it’s critical to get faculty input via usability studies. Redesign of the Libraries’ website….
They will get it, we’re only one piece of the puzzle. Is that ok with us? Our mission to the University? If they’re spending grant/research money on this, what is the library doing?
We can do this. Easily.
Also, “discipline” mentality is dominant over “status” mentality.
For me, this is my goal for our collections. What I’m always thinking about. How can we make them as easy to access as possible. We have so much – EVERYTHING! They know it, we know it. They can’t always get to it.
That’s the difference in perspective btwn library staff and faculty.
Obviously, we didn’t include historians, but there were plenty of humanists. Avery faculty were the minority print-preferers.
Evolving attitudes, as the quality of digital content improves. It seemed to be that as more backfiles are purchased and PDFs are more commonly available, that acceptance of e-journals is increasing. The convenience of e- was nearly consistently praised.
I thought this was going to be the million dollar issue (literally). Expected this to be a much larger problem. This was my biggest surprise. Maybe they’ve managed to figure it out, on the whole.
This concept of “service” intrigued me, when you juxtapose it with in-person service one receives at the library.