Collaborate, Evaluate, Evolve CILIP Ireland/ LAI Annual Joint Conference, 8-10 April 2015, Titanic Belfast

1. Virginia Power,
Graduate Tutor/Researcher in Information Management & Science,
University of the West of England, Bristol
Some statistics provided by Carol Parker, Barton Peveril College, Hampshire

2. Drawing on research from the UK, the
US and Scandinavia, the LISU study
shows a world in which libraries are
actively engaged in gathering evidence
to demonstrate their value – but
challenges remain. Though a good
deal of evidence is collected, much of
it is evidence of activity rather than
evidence of value and impact.
Barr, S. (2012) How should academic
libraries communicate their own value?
Guardian Online
http://www.theguardian.com/higher-education-
network/blog/2012/aug/20/academic-libraries-
value-research-teaching
Evolving Value for Libraries

4. Your motivation. Identify the key reasons why you want to
engage in a proving and/or improving process, your expectations
about the process, ‘windows of opportunity’, the available
resources, leadership issues and the appropriate tools or
methods.
Your vision. Set forth the organisation’s mission, values, and
objectives. Identify its major activities.
Stakeholders. Know who is affected by or affects your
organisation in all of its activities, whether intentionally or not.
Impact Map. Specify what the organisation does and how it does
it; how it intends to achieve its social, environmental/economic
mission as well as its financial sustainability; and how it plans to
live up to its values.
Measuring the Library

5.  Identifying how well a service fulfils its purpose
and adds value to an organisation
 Accountability and justification of financial
spending
 Identifying areas for improvement
 Monitoring progress against goals
 Comparing standards with other organisations
 Monitoring effects of past decisions
 Monitoring interactions with customers and
suppliers

6. Have YOU read and understood the
inspection framework for your institution?

7. ‘Distance travelled’ is the description given to the change in ‘soft
outcomes’, that is, those achievements that cannot be directly
measured in the same way as ‘hard outcomes’ such as
qualifications, earnings or employment status. Most
simply, ‘distance travelled refers to the progress that a
beneficiary makes towards employability or harder outcomes’. It
is easy to measure an individual’s educational progress, for
example, as this can be a number of qualifications or a
jump from level 2 to level 3. It is much more difficult to measure
progress in social skills or motivation, although these are as
important for an individual’s life chances.
Soft Outcomes for Universal Learning (SOUL)
Are YOU able to demonstrate the distance
travelled by the students that you work with
in Library & Information Services?

9.  The library needs to be able to identify and
show progress
 Identify both intended and unexpected
outcomes.
 Prioritise what is most important to measure
 Choose the ways in which you will see change
happening.

12.  Choose KPIs that align with service and
organisational aims/objectives
 Review KPI annually
 How will they be measured/monitored?
 Choose quality over quantity
 Adopt a range of methods to measure
performance
 How much time/money/tools do you have to
measure/monitor PI?

13. Make a plan. After prioritising the indicators, decide on methods
for collecting information relating to the indicators.
Collect data. Collecting information can be done in a number of
ways as outlined in your methodology or plan, as long as it can
be stored and drawn upon usefully.
Analyse information. Seeing what the data you collect tells you
can happen throughout the process, and can also be summed up
at the end of a cycle to make some conclusions.

14. You are trying to understand
Whether, how and what impact occurs
What the outcome of this impact is for the people concerned
What the broader impact of this evidence might be
Before you begin to collect evidence decide
Why you are seeking this evidence
What kind of information you will collect
Who will collect it
When it will be collected
How to use the results
What do you want to find out? Keep it
simple and focused
What are the learning outcomes of the
information skills session?
What new knowledge did students
learn and how did this knowledge
increase their understanding?
How did using the library change
attitudes towards libraries and
learning?

16. Review the data you collect already
Comment books
Teacher's evaluation sheets
Letters from users
Footfall counts
User surveys
Project evaluation forms
Other...
For each data source, ask the question: does it give evidence of impact?
Yes
No
Maybe?

17. 0 2 4 6 8 10 12 14 16 18 20
Discussion at meetings
Comments and suggestions
Feedback forms
Staff voice meetings
One-to-one sessions
Surveys
Boards of study
Annual survey
Liaison role
Course committees
Focus groups
Percentage of respondents
Feedback methods
Q.4 Feedback methods from internal users

18. Most Effective Method for Gathering User Feedback Percentage of respondents
Notice boards 9.0
Student union 23.9
Social media 26.9
Student council 26.9
National surveys 29.9
Online feedback mechanisms 35.8
Organisational/institutional surveys 44.8
Formal complaints/feedback procedures 47.8
Focus groups 53.7
User surveys 82.1

20. Library managers need to ensure that they
recognise the concept of the internal customer
within their organisation. This is because
“errors in the service provided within an
organisation will eventually affect the product
or service which reaches the external
customer” (Slack et al, 2004, p. 723).

22. TICSI (no date) states that benchmarking
encourages a service to “become open to new
methods, ideas, processes and practices that
improve effectiveness, efficiency and
performance in customer service”
 Further Education – CoLRIC
 Higher Education and Research – SCONUL and
National Student Survey

23.  Balanced scorecard
 CoLRIC – performance and impact indicators
/ peer accreditation scheme
 EFQM – European Foundation for QM
 Inspiring Learning for All
 SCONUL benchmarking
 LibQUAL+
 Prove and Improve Comparison Chart
 Return on investment – ROI Calculator

24. Business Processes
Streamlining processes
New products/suppliers
Supply chain
Staff skills and expertise
Training needs
Customer satisfaction
Meeting customer needs
Cash flow
Costs
Budgets
Balanced scorecard

25. Olson (2002, p.19) states
“information services are
considered valuable when they are
perceived to be equivalent to the
funds spent on them”.
ROI Calculator

26. Library Performance Measurement in the UK
and Ireland
Association of Research Libraries, 2009

27. Sharing information. Make what you have learnt a part of a
dialogue or conversation, offering different stakeholders the
opportunity to learn from the process and feed their thoughts
and opinions into the next round.
Learning & Action. A proving and/or improving process is a
chance to do things better in the future. Make what you have
learnt part of a plan for improvement –the necessary actions,
changes and the resources needed to enable the desired
changes.

29. Service level agreement
Annual report
Quarterly report
Self assessment review
Business report
0 10 20 30 40 50 60
Methods of
Communication
Percentage of respondents
Q.7 How do you communicate the results of performance
measurement?

30. Performance measurement and impact assessment is time
consuming and difficult to do.
Anecdotal evidence suggests that even when mechanisms
are put in place to measure performance e.g. feedback and
customer satisfaction - it isn't always forthcoming!
Services need to use a variety of methods to get good
results.
Library Managers need a range of skills to support this.

32.  Communication Skills
 Marketing Skills
 Promotional Tools
 Systematic collection of evidence
 Analytical skills
 Social Informatics
 Understanding of big data

33.  The role of data informing decision-making
 New approaches to collecting, analysing and
using data
 Using analytics to develop new services and
improve the user experience
 The opportunities of library data as big data
 Role of library analytics uncovering new
opportunities to demonstrate impact and
value

34. Virginia Power,
Graduate Tutor/Researcher in Information Management & Science,
University of the West of England, Bristol
Virginia.Power@uwe.ac.uk