June 18, 2014 NISO Virtual Conference: Transforming Assessment: Alternative Metrics and Other Trends What Do Alternative Metrics and the 'Flight of the Bumblebee' Have in Common? - Martha Kyrillidou, PhD, Senior Director of Statistics and Service Quality Programs, Association of Research Libraries (ARL)

1. What Do Alternative Metrics and
the 'Flight of the Bumblebee'
Have in Common?
Martha Kyrillidou
ARL Statistics and Service Quality Programs
Association of Research Libraries (ARL)
NISO June Virtual Conference:
Transforming Assessment: Alternative Metrics and Other Trends
June 18, 2014

2. e-Pubs downloads (May 16, 2014)
“I made the Libraries presentation to the Purdue Board of
Trustees this morning, including a video of the downloads from
Purdue e-Pubs, speeded up to show downloads in 2 hours in 30
seconds (nearly 3,000). The sound track was “Flight of the
Bumblebee.” The greatest compliment I had from a Trustee
after the presentation was when he said, his only complaint was
“that the presentation didn’t go longer, he found it very
interesting, totally unexpected!”
James L. Mullins, PhD
Dean of Libraries and
Esther Ellis Norton Professor
Purdue University

3. ENABLING WORLD-CHANGING RESEARCH
Libraries’ Purdue e-Pubs
As of May 8, 2014
37,580
documents
6,908,130
total downloads
2,596,249
past year

4. Transformation I
• "Flight of the Bumblebee" is an orchestral
interlude written by Nikolai Rimsky-Korsakov for
his opera The Tale of Tsar Saltan, composed in
1899–1900. The piece closes Act III, Tableau 1,
during which the magic Swan-Bird changes
Prince Gvidon Saltanovich (the Tsar's
son) into an insect so that he can fly
away to visit his father (who does not
know that he is alive).

5. Association of Research
Libraries
ARL Statistics and Assessment
…To describeand measure the
performance of research
librariesand their contribution to
teaching, research, scholarship and
community service …

6. Association of Research
Libraries
ARL Statistics and Assessment
…To describe and measure the performance of
research libraries and their contribution
to teaching, research,
scholarshipand community service
…

7. The Printed Library Materials I Need
for My Work

8. Print and/or Electronic Journal Collections I
Require for My Work
www.libqual.org

9. A Library Website Enabling Me to Locate
Information on My Own

10. StatsQUAL®
A gateway to library assessment tools that describe the
role, character, and impact of physical and digital libraries.
LibQUAL+® is a
rigorously tested Web-
based survey that
libraries use to solicit,
track, understand, and
act upon users‘ opinions
of service quality.
LibQUAL+® MINES
for Libraries®
Measuring the Impact of
Networked Electronic
Resources (MINES) is an
online transaction- based
survey that collects data on
the purpose of use of
electronic resources and
the demographics of users.
ClimateQUAL®
ClimateQUAL® :
Organizational Climate
and Diversity Assessment
is an online survey that
measures staff perceptions
about: (a) the library's
commitment to the
principles of diversity, (b)
organizational policies and
procedures, and (c) staff
attitudes.
ARL
Statistics®
ARL Statistics™ is a series
of annual publications that
describe the collections,
expenditures, staffing, and
service activities for
Association of Research
Libraries (ARL) member
libraries.
LibValue
LibValue is a series of
tested methods that
capture Return on
Investment and Value of
library services.

11. The data and the models
• I hesitate to hang the future of libraries on a
demonstrable effect size for the equation,
L = $$$
– Rachel Applegate , Indiana University SLIS
ARL
Statistics®
ARL Statistics™ is a series
of annual publications that
describe the collections,
expenditures, staffing, and
service activities for
Association of Research
Libraries (ARL) member
libraries.

12. Why study value
• Faced with difficult economic times and
university budget cuts, the value of the library
to the wider goals of the university is
increasingly questioned.
– Introduction to the Lib-Value Project
» www.libvalue.org
Association of Research
Libraries

13. Association of Research
Libraries
Use ……Value

14. MINES for Libraries –
Scholars Portal @ OCUL
9th Northumbria Conference, 23rd August 201114
In the Library
Off-campus
On-campus but
not in the library
In the Library Off-campus On-campus but not in the library

15. LibValue
Special Collections – Google Analytics application
• LibValue: Digitized Special Collections (video on YouTube)
Thursday, August 15, 2013, 1:00–2:00 p.m. eastern
Presenters:
Gayle Baker, Professor and Electronic Resources Coordinator,
University of Tennessee Libraries
Ken Wise, Associate Professor, University of Tennessee Libraries
www.arl.org

16. A Values scorecard
• The value which stakeholders seek from libraries
is transcendent, in the sense that the impact
demonstrated must be beyond the library and
immediate satisfaction or fulfillment. Library
assessment effort to date has been (mainly)
about quality rather than value.
• J. Stephen Town, Martha Kyrillidou, (2013) "Developing
a values scorecard", Performance Measurement and
Metrics, Vol. 14 Iss: 1, pp.7 - 16
http://www.emeraldinsight.com/journals.htm?articleid
=17086961
Association of Research
Libraries

17. A Value Scorecard
• Relationship & reputation capital
• Organisational capital
– Tangible assets and resources
– Intangible and meta-assets
• Library virtue
– Impact and social capital
• Library momentum …. Leadership … Culture

18. Article-level metrics –
Relation/Reputation
• Usage
• Captures
• Mentions
• Social Media
• Citations
– Tananbaum, Greg. (2013, April 16). Article-level metrics: a SPARC
primer. Scholarly Publishing and Academic Research
Coalition. http://www.sparc.arl.org/sites/default/files/sparc-alm-
primer.pdf
– Bollen, J. et al. (2008). Towards alternative metrics of journal
impact: A comparison of download and citation data.
http://arxiv.org/pdf/cs.DL/0503007.pdf

20. ARL Strategic Thinking and Design
• By 2033 the research library will have shifted
from its role as a knowledge service provider
within the university to become a
collaborative partner within a rich and diverse
learning ecosystem

21. System of action components
– Coordinated management of collective collections
– Scholarly publishing at scale
– ARL Academy
– Building a boundless symposium
– A first suite of smart libraries
– Innovation lab

22. Transformation II
Questioning
assumptions
Shaping the
experience

23. La danse (I) by Matisse, 1909, The Museum of Modern Art, New York City

24. Henri Matisse The Dance, 1910, Hermitage Museum, St. Petersburg, Russia

25. Dancing to the end of Love
Henri Matisse Music, 1910, Hermitage Museum, St. Petersburg, Russia

26. Why I Love ….
• Stay connected
• Get coffee
• Take a break
• Check out my class book
• Reserve a conference room for a meeting
• Use my own computer and plug in
• Use the library’s computers
• Study with a friend or on my own
• Share laughs and hang with friends
• Studying

27. Final Blessings
• Indulge your curiosity
• Influence your friends
• Enable your organization.
– May your analytics always be good – and your
judgment even better.
» Robert Morison, Analytics at Work