Medical Library Association Annual Meeting, May 2012, Seattle Lorie Kloda, McGill University Denise Koufogiannakis, University of Alberta Alison Brettle, Univerity of Salford

1. Lorie Kloda, MLIS, PhD(c), AHIP, McGill University
Denise Koufogiannakis, MA, MLIS, PhD(c), University of Alberta
Alison Brettle, MLIS, PhD, University of Salford
Medical Library Association Annual Meeting, May 2012, Seattle

2. Gap between research and practice in library and
information studies (LIS)
(Booth, 2003; Crowley, 2005; Genoni, Haddow, & Ritchie, 2004; Turner, 2002)
Only method likely to improve communication is
“inclusion of research reports in (…) publications
frequently read by practitioners.”
(Haddow&Klobas, 2004)
Evidence Based Library and Information Practice
journal, 2006-
>200 evidence summaries

3. Structured abstract
objective – design – setting – subjects –
method – main results – conclusion
Commentary
 300-400 words
 appraisal of validity, reliability, applicability
 significance, implications for practice
3

4. 4

5. To investigate the impact of evidence
summaries on library and information
professionals and their practice
Knowledge
Practice
Users

6. Phase 1
Development and face-validation of tool
Phase 2
Survey questionnaire to readers (QUANT)
Phase 3
Interviews (QUAL)

7. Development of a tool to assess impact

8. Findings (Phase 1)
Development of
Impact
Assessment
Method
Grad, R., Pluye, P., &
Beauchamp, M.-E.
(2007). Validation of
a method to assess
the clinical impact of
electronic
knowledge
resources, e-Service
Journal, 5(2), 113-
135.
http://iam2009.pbworks.com/

9. Survey questionnaire to readers of evidence summaries

10. Findings (Phase 2)
Survey
175 153 emailed 101
Respondents recruited IAM survey respondents
1 unusable 15
3 bounced
email incomplete
86
21 duplicates 49 no reply completed
IAM survey
153
remaining
101 total
respondents n=62

11. Findings (Phase 2)
USA
Country
Canada
UK
Australia
USA Ireland
Australia Finland
Malaysia
UK Saudi Arabia
Iran
Puerto Rico
Spain
Canada
Brazil
Hong Kong

12. Findings (Phase 2)
50+
25+
Number of 15
Evidence 14
13
Summaries 12
11
Read in Past 10
9
Year 8
7
6
5
4
3
2
1
0
0 2 4 6 8 10
No. of respondents

13.  Decline in Reference Transactions with Few Questions Referred to
Librarian when the Reference Desk is Staffed by a Paraprofessional (8)
 The Presence of Web 2.0 Applications Is Associated with the Overall
Service Quality of Library Websites (6)
 Google Scholar Out-Performs Many Subscription Databases when
Keyword Searching (4)
 Statistical Measures Alone Cannot Determine Which Database
(BNI, CINAHL, MEDLINE, or EMBASE) Is the Most Useful for Searching
Undergraduate Nursing Topic (4)
 A Graduate Degree in Library or Information Science Is Required, but not
Sufficient, to Enter the Profession (3)

14. Findings (Phase 2)
Reason for Freq. %
Reading For general interest or curiosity 15 24%
Evidence
Summary For personal continuing 18 29%
professional education
n=62 To answer a specific question 21 34%
or address a specific issue in
my practice
Other 8 13%

15. Findings (Phase 2)
When the > 1 year
Evidence 6-12 months
Summary was
3-6 months
Read
1-3 months
1 week-1 month
1-7 days
Today
0 5 10 15 20 25

16. Findings (Phase 2)
Freq. %
Cognitive
My practice was (will be) improved 11 13%
Impact I learned something new 36 42%
I recalled something I already knew 14 16%
It prompted me to investigate more 23 27%
It confirmed I did (I am doing) the right thing 17 20%
I was reassured 13 15%
I was dissatisfied: There is a problem with the 1 1%
presentation of this evidence summary
I was dissatisfied: I disagree with the content 0 0%
of this evidence summary
It is potentially harmful 0 0%
Other 9 10%

17. Findings (Phase 2_
Cognitive freq. %
Impact:
Too much information 0 0%
Presentation
Problem Not enough information 1 1%
“You reported: I was Information is poorly written 0 0%
dissatisfied; There was
a problem with the
presentation of this
Information is too technical 0 0%
Evidence Summary.
Which of the following Other 0 0%
problems did you
encounter?”

18. Findings (Phase 2)
Freq. %
Cognitive
The structured abstract did not 0 0%
Impact: adequately explain the original study
Disagree with
The writer of the evidence summary 0 0%
Content presented incorrect information
“You reported: I was The commentary was overly negative 0 0%
dissatisfied; I disagree
with the content of this The commentary was not critical 0 0%
Evidence enough
Summary. Which of the The writer of the evidence summary did 0 0%
following content
not place this study in context
elements did you
disagree with?” Other 0 0%

19. Findings (Phase 2)
Practice Freq. %
Impact Change my service approach 5 6%
“You reported: Change my approach to collections 1 1%
My practice was
(will be) Change my management approach 4 5%
improved. What
Change my approach to teaching 4 5%
did you (will you)
do differently Change my professional approach 4 5%
after reading the
Evidence Other 3 3%
Summary?”

20. Findings (Phase 2)
Community  None
Impact
 Hypothesized future/potential
“If reading this
Evidence Summary impacts on users
resulted in some  Reinforced cognitive or practice
change to your
individual practice, do
you think it led to an
impacts, not user outcomes
impact on anyone  5 reported actual impact at this
within the community
you serve or level:
environment in
which you work?  Change in teaching LIS students
Please explain in the
comment box.”  Observed (anecdotal) changes

21. Interviews with subsample of survey respondents

22. Findings (Phase 3)
13 participants Subsample from survey
respondent
Various settings, education levels,
countries, and professional
positions

23. Proposed Impacts Suggested Confirmed
Phase 1 Phase 2 Phase 3
C1 Practice was improved ? ✗
C2 Learned something new ✔ ✔
C3 Recalled something ✔ ✓
C4 Prompted to investigate ✔ ✔
C5 Confirmed ✔ ✓
C6 Reassured ✔ ?
C7 Dissatisfied – presentation ? 0
C8 Dissatisfied - content 0 0
C9 Potentially harmful 0 0
P1 Service ✔ ✔
P2 Collections ✔ ✓
P3 Management ✔ ✓
P4 Teaching ✔ ✓
P5 Professional practice ✔ ✓
U User Community ? ?

24. Findings (Phase 3)
Potential Discovery
 New research
Impacts  Interesting topics
 Methods
Uncovered  Keeping current
Sharing
 With colleagues, managers
 Report writing
 Recommended reading
Assistance
 Research
 Writing, presentations
 Teaching (for professors)

25. Findings (Phase 3)
“Likes” Format of evidence summaries
 Descriptive
 Accuracy of abstract
 Concise
Time saving
Database of Evidence Summaries

26.  One evidence summary assessed per
respondent
 Cognitive impact comparable to findings in
Grad, Pluye, et al. (2006)
 Practice impact – two-tiered
 Low community impact

27.  Tool validation
 Cognitive impact
 Practice impact
 Individual practice
 Workplace practice
 Difficult to assess impact on
community/users

28. Revise Impact Assessment Tool
 Include new or revised impact items
 Improve wording of items
Conduct survey of a larger sample of evidence
summary readers
Adapt tool for other forms of research
dissemination

29. Canadian Association of Research Libraries
Research in Librarianship Grant
Roland Grad and Pierre Pluye, McGill University,
for their feedback
All of our survey respondents and interview
participants

30. lorie.kloda@mcgill.ca
@loriekloda
@eblip
slideshare.net/lkloda