Project output versus influence in practice: impact as a dimension of research quality

Project output versus influence in
practice

Impact as a dimension of research
quality

6th International Conference on Evidence based Librarianship and Information
Practice
Keynote presentation by Professor Hazel Hall, Director, Centre for Social
Informatics, 30th June 2011

http://lisresearch.org
http://lisresearch.org

If you are tweeting this
If you are tweeting this
session, please refer to me
session, please refer to me
as @hazelh. Thank you!
as @hazelh. Thank you!

Key points

 LIS impact takes a number of forms
 Research impact measurement is tricky
 Research impact measurement matters
 Where impact is measured as a dimension of
research quality, a number of strategies can be
deployed to extend project outcome reach

Impact themes in LIS literature
1. Evaluation of services – general use (debated)
1. Evaluation of services – general use (debated)

2. Evaluation of service impact on end user behaviours
2. Evaluation of service impact on end user behaviours

Impact themes in LIS
Impact themes in LIS
literature
literature
3. Evaluation of services as
3. Evaluation of services as
related to specific
related to specific
outcomes
outcomes

4. How to evaluate services
4. How to evaluate services

5. Bibliometrics
5. Bibliometrics

5. Bibliometrics – publication impacts
5. Bibliometrics – publication impacts

6. Bibliometrics research evaluation
6. Bibliometrics research evaluation
We can stand on the shoulders of giants
We can stand on the shoulders of giants
because librarians and information
because librarians and information
scientists have maintained a curated record
scientists have maintained a curated record

7. LIS research output  LIS practice
7. LIS research output  LIS practice

8. LIS practice  LIS research
8. LIS practice  LIS research
(Brookes, 1976)
(Brookes, 1976)

Impact routes vary
Impact routes vary

Cumulative and indirect effect of
Cumulative and indirect effect of
exposure to research output
exposure to research output

Direct policy mandate that
Direct policy mandate that
derives from a particular study
derives from a particular study

Key points

 Research impact matters
research quality, a number of strategies can
be deployed to extend project outcome reach

Measurement challenge 1
Attribution, and cause and effect, are
not always accurate nor obvious

Greer Garson
Greer Garson
Best actress 1942 for role in Mrs Miniver
Best actress 1942 for role in Mrs Miniver
Acceptance speech 5 minutes 30 seconds
Acceptance speech 5 minutes 30 seconds


Sources of mediated training material are
Sources of mediated training material are
not always appreciated by practitioners
not always appreciated by practitioners

End-users are not normally aware of research
End-users are not normally aware of research
effort to support services design and delivery
effort to support services design and delivery

Certain types of research output attract more
Certain types of research output attract more
attention than others
attention than others

The impact of some is overwhelming
The impact of some is overwhelming
(and disproportionate)
(and disproportionate)

Key points

 Research impact matters

Practitioner interest in research output
Practitioner interest in research output
Increases value of research undertaken
Increases value of research undertaken
Narrows research-practice gap 
Narrows research-practice gap 
benefits health of the profession
benefits health of the profession

Local contexts influence attitudes
Local contexts influence attitudes
towards impact measurement
towards impact measurement
REF2014 in the UK
REF2014 in the UK

NB presented here are strong
NB presented here are strong
messages that have emerged from
messages that have emerged from
RiLIES project empirical data
RiLIES project empirical data
analysed to date
analysed to date

http://lisresearch.org/rilies-project/
http://lisresearch.org/rilies-project/

Literature review; poll; focus
Literature review; poll; focus
groups; case studies; survey
groups; case studies; survey

Research ideas for impact in practice
Research ideas for impact in practice

Focus on research areas of direct relevance
Focus on research areas of direct relevance
to practitioners, both in terms of subject
to practitioners, both in terms of subject
and context -- time, place, sector
and context time, place, sector

(Perth and London focus groups)
(Perth and London focus groups)

Research approach for impact in practice
Research approach for impact in practice

Deploy action research approach
Deploy action research approach

Mix in development and engagement, e.g. capacity
Mix in development and engagement, e.g. capacity
building as hook, workshops throughout project
building as hook, workshops throughout project

Encourage community creation around project for
Encourage community creation around project for
sustainability
sustainability

Involve high profile, prestigious partners
Involve high profile, prestigious partners

(Evalued, Evaluating the impact of clinical librarian
(Evalued, Evaluating the impact of clinical librarian
services, Open to all case studies)
services, Open to all case studies)

Project scale and diversity for impact in practice
Project scale and diversity for impact in practice

Data collection needs to be on a scale for project
Data collection needs to be on a scale for project
recommendations to be applicable elsewhere
recommendations to be applicable elsewhere

Ensure that projects are outward-looking to
Ensure that projects are outward-looking to
safeguard against accusations of bias
safeguard against accusations of bias

(Perth focus group)
(Perth focus group)

Face-to-face dissemination for impact in practice

Exploit face-to-face dissemination as the practitioners’
Exploit face-to-face dissemination as the practitioners’
favoured route
favoured route

Target dissemination according to career stage preferences
Target dissemination according to career stage preferences

Invest project budget in training activities for project legacy
Invest project budget in training activities for project legacy

Launch events effective for policy maker target audiences
Launch events effective for policy maker target audiences

(Poll, focus groups, case studies)
(Poll, focus groups, case studies)

Accessible textual sources for impact in practice
Accessible textual sources for impact in practice

Tailor a variety of reports according to intended audiences:
Tailor a variety of reports according to intended audiences:
tools they can access, career stage, sector…
tools they can access, career stage, sector…

Make reports accessible: easy to locate e.g. open access;
Make reports accessible: easy to locate e.g. open access;
understandable, e.g. possible to act on project outcomes
understandable, e.g. possible to act on project outcomes

Work with profile dissemination partners (e.g. HEFCE, JISC,
Work with profile dissemination partners (e.g. HEFCE, JISC,
OCLC, Pew, RIN, SCONUL) to enhance authority of work and
OCLC, Pew, RIN, SCONUL) to enhance authority of work and
increase its reach
increase its reach

Ensure work is picked up by digest/alert services and bloggers
Ensure work is picked up by digest/alert services and bloggers

http://www.twitter.com/LISResearch
http://www.twitter.com/LISResearch
provides an alert services
provides an alert services

Conference registration open

Training day announcement
Training day announcement

Publication of research report
Publication of research report

Study day announcement

Conference registration to close
Conference registration to close



Publication of blog post
Publication of blog post

Exploit network hubs
Exploit network hubs
5 key individuals in UK
5 key individuals in UK
emerge from RiLIES project…
emerge from RiLIES project…

5 UK research connectors
1. Rhona Arthur @RhonaEA
1. Rhona Arthur @RhonaEA

2. Andrew Booth @scharrlock
2. Andrew Booth @scharrlock

3. Phil Bradley @Philbradley
3. Phil Bradley @Philbradley

4. Alison Brettle @BrettleAli
4. Alison Brettle @BrettleAli

5 UK research
5 UK research
connectors
connectors
5. Hazel Hall @hazelh
5. Hazel Hall @hazelh


What happens when formal opportunities for face-to-
What happens when formal opportunities for face-to-
face dissemination are restricted:
face dissemination are restricted:

training/travel budgets cut?
training/travel budgets cut?

reduced professional memberships?
reduced professional memberships?

… and when research infrastructures are fractured? This
… and when research infrastructures are fractured? This
has been addressed in UK in the formation of the
has been addressed in UK in the formation of the
Library and Information Science Research Coalition
Library and Information Science Research Coalition

Researcher priority
Researcher priority
Present research output in an accessible
Present research output in an accessible
way
way

Service manager priority
Service manager priority
Facilitate research output consumption,
Facilitate research output consumption,
e.g. reverse mentoring, job descriptions,
e.g. reverse mentoring, job descriptions,
reporting back from conferences
reporting back from conferences

Course/accreditation priority
Course/accreditation priority
Application of research in practice
Application of research in practice

A greater role for social media?
A greater role for social media?

Currently divides the practitioner
Currently divides the practitioner
community, e.g. blogs very popular with
community, e.g. blogs very popular with
academic librarians but inaccessible to
academic librarians but inaccessible to
some others; Twitter essential to some,
some others; Twitter essential to some,
irrelevant to others
irrelevant to others

http://www.bite.ca/bitedaily/2010/07/social-media-monopoly-board-game/

A greater role for self-organising
A greater role for self-organising
networks as communities of practice?
networks as communities of practice?

(with technical infrastructure provided by
(with technical infrastructure provided by
social media)
social media)

Research funding to support researcher-
Research funding to support researcher-
practitioner links
practitioner links

http://www.lisresearch.org/dream-project
http://www.lisresearch.org/dream-project

DREaM project launch conference
DREaM project launch conference
Tuesday 19th July, London
Tuesday 19th July, London

Places still available
Places still available

Key points

 Research impact measurement matters
 Preliminary findings of RiLIES project dominated
by influence of dissemination strategy

Bringing it back to Salford:
anyone recognise these lads?

What difference does it make?
(1984)

Playlist 20 March 1984
Playlist 20 March 1984

Hand in glove; Still ill; Heaven knows I’m
Hand in glove; Still ill; Heaven knows I’m
miserable now; This charming man; Girl afraid;
miserable now; This charming man; Girl afraid;
Pretty girls make graves; This night has opened
Pretty girls make graves; This night has opened
my eyes; Barbarism begins at home; What
my eyes; Barbarism begins at home; What
difference does it make? II don’t owe you
difference does it make? don’t owe you
anything; Handsome devil; You’ve got
anything; Handsome devil; You’ve got
everything now; These things take time
everything now; These things take time

Photo credits

Image: 'I Can Gather All the News I+Need+on+the+Weather+Report' Image: 'Greer Garson & Robert Young'
http://www.flickr.com/photos/51035555243@N01/2671536366 http://www.flickr.com/photos/44186100@N00/2490143154

Image: 'These Books are Sweet' Image: ‘Waiting for the library to open’
http://www.flickr.com/photos/47823583@N03/4474869956 http://www.flickr.com/photos/davepattern/1424246518/in/set-
72157602122007409
Image: ‘Radar Graph – Library Usage v Final Grades'
http://www.flickr.com/photos/davepattern/4172110833/in/set-72157622847182901/ Image: 'BC007'
http://www.flickr.com/photos/51548378@N05/4745027706
Image: '50lb weights'
http://www.flickr.com/photos/62327186@N00/2185667691 Image: 'Money to burn'
Image: 'Article-level metrics at PLoS'
http://www.flickr.com/photos/14829735@N00/3929622080 Image: 'Warhol's Light Bulbs'
Image: '365: day 119'
http://www.flickr.com/photos/33563858@N00/2451800342 Image: 'Cheers'
Image: 'The locker misery'
http://www.flickr.com/photos/33685308@N00/513100970 Image: 'P1010121'
Image: '90" engine valves'
http://www.flickr.com/photos/61132483@N00/2975215 Image: 'My social Network on Flickr, Facebook, Twitter+and+MyblogLog'
Image: 'Oscar'
http://www.flickr.com/photos/45756179@N03/5068766020 Smiths at Salford Lads Club (inside cover of The Queen is dead)
by Stephen Wright
Image: 'ed vaizey'

Reference & acknowledgements

 Brookes, B.C. (1976). A new paradigm for
information science. The Information Scientist,
10(3), 103-12.
 Peter Cruickshank, Ella Taylor-Smith, Jenny
Gebel (RiLIES project team)
 Stephanie Kenna (LIS Research Coalition)
 All those who have contributed data to date

Professor Hazel Hall
h.hall@napier.ac.uk @hazelh

Project output versus influence in practice: impact as a dimension of research quality

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a Project output versus influence in practice: impact as a dimension of research quality

Semelhante a Project output versus influence in practice: impact as a dimension of research quality (20)

Mais de Hazel Hall

Mais de Hazel Hall (20)