Getting Evidence into Practice
•
1 gostou•912 visualizações
Quebec Library Association Annual Meeting, Montreal, May 9, 2013 Closing Keynote
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Destaque
Destaque (7)
Semelhante a Getting Evidence into Practice
Semelhante a Getting Evidence into Practice (20)
Mais de Lorie Kloda
Mais de Lorie Kloda (12)
Último
Último (20)
Getting Evidence into Practice
- 1. Getting Evidence into Practice Lorie Kloda McGill University Quebec Library Association May 9, 2013
- 2. 1 Assessment
- 3. Two Solitudes? Are these separate movements within librarianship forming theoretical bridges? Is some sort of merger, fusion or takeover in the future? Ryan, P. (2006). EBL and library assessment: Two solitudes? Evidence Based Library and Information Practice, 1(4), 77–80. “ ” 2
- 5. to assess, in general, is to determine the importance, size, or value of; to evaluate. Library staff assess operations by collecting, interpreting, and using data to make decisions and to improve customer service. They study internal processes, levels and quality of service, and library impact on institutional goals. Wright, S., & White, L. (2007). Library Assessment, SPEC Kit 303 (Washington, DC: Association of Research Libraries, Office of Leadership and Management Services). “ ” 4
- 6. includes any activities that seek to measure the library’s impact on teaching, learning and research as well as initiatives that seek to identify user needs or gauge user satisfaction or perceptions with the overall goal being the data‐based and user‐centred continuous improvement of our collections and services. Ryan, P. (2006). EBL and library assessment: Two solitudes? Evidence Based Library and Information Practice, 1(4), 77–80. “ ” 5
- 9. 8
- 10. What is evidence-based library and information practice?
- 11. an approach to information science that promotes the collection, interpretation and integration of valid, important and applicable user-reported, librarian observed, and research- derived evidence. The best available evidence, moderated by user needs and preferences, is applied to improve the quality of professional judgements. Booth, A. (2000). Librarian heal thyself: Evidence based librarianship, useful, practical, desirable? Proceedings from the 8th International Congress on Medical Librarianship. London, UK, July 2- 5, 2000. “ ” 10
- 12. EBLIP Process Formulating the Question Searching the Evidence Appraising the Evidence Applying the Results Evaluating the Process 11
- 13. Getting research evidence into practice
- 15. 14
- 16. Evidence Summaries http://ejournals.library.ualberta.ca/index.php/EBLIP Evidence Based Library and Information Practice journal, 2006- >250 evidence summaries
- 18. 17
- 19. 18
- 20. 19
- 21. 20
- 22. 21
- 24. 23
- 26. EBLIP Formulate Search Appraise Apply Evaluate Information Literacy Know Access Evaluate Use Ethical/Legal 25 American Library Association. (2006). The standards: Step-by-step URL: http://www.ala.org/ acrl/issues/infolit/standard s/steps
- 29. Evolution of EBLIP Movement 1992 EBM coined at McMaster 1994 HainesHILJ editorial 1997 EldredgeHypothesis (MLA) article 2000 Eldredge EBL framework series 2001 First EBL (now EBLIP) conference 2004 Booth& Brice book 2006 EBLIP journal launches 2013 Marshall MLA keynote 28
- 30. 29
- 31. Local Evidence User feedback Librarian observation Discussion/interactions with colleagues Usage data Organizational realities 30
- 32. Professional Knowledge Formal and informal learning On the job training Tacit knowledge Reflective knowledge 31
- 33. 32 Koufogiannakis, D. (2010). Considering the place of practice-based evidence within evidence based library and information practice (EBLIP). Library and Information Research, 35(111), 41–58.
- 34. How does evidence influence librarianship and librarians?
- 35. Impact of Published Evidence 34 Kloda, L., Koufogiannakis, D., &Brettle, A. Assessing the impact of evidence summaries in library and information studies: A mixed methods approach. Speaker. Health Libraries Group Conference, Glasgow, Scotland, July 11- 12, 2012.
- 36. Getting assessment evidence into practice
- 37. 36
- 38. 37
- 39. 38
- 40. 39
- 41. 40 Gerlich, B. K., &Berard, G. L. (2010). Testing the viability of the READ scale (Reference Effort Assessment Data): Qualitative statistics for academic reference services. College and Research Libraries, 71(2), 116–137.
- 42. Information Literacy Instruction Assessment Cycle 41 Oakleaf, M. (2009). The Information Literacy Instruction Assessment Cycle: A guide for increasing student learning and improving librarian instructional skills,” Journal of Documentation, 65(4) 539–560.
- 43. How does EBLIP fit into assessment? Or is it the other way around?
- 44. Two Solitudes? Are these separate movements within librarianship forming theoretical bridges? Is some sort of merger, fusion or takeover in the future? Ryan, P. (2006). EBL and library assessment: Two solitudes? Evidence Based Library and Information Practice, 1(4), 77–80. “ ” 43
- 45. Widening the EBLIP Model Articulate the problem Assemble evidence Appraise for quality & quantity Agree on the way forward Adapt 44
- 49. A culture of assessment is an organizational environment in which decisions are based on facts, research, and analysis, and where services are planned and delivered in ways that maximize positive outcomes and impacts for customers and stakeholders. Lakos, A., Phipps, S., & Wilson, B. Defining a ‘Culture of Assessment.’ (1998– 2002), http://www.library.ucla.edu/yrl/reference/aalakos/assess ment/CulAssessToolkit/ Assessdef3-new.doc “ ” 48
- 50. Questions 1 2 49
- 51. Questions How can we foster EBLIP in our work or in our workplace? 2 50
- 52. Questions How can we foster EBLIP in our work or in our workplace? What are some ways to overcome obstacles to to using evidence and doing assessment? 51
Notas do Editor
- I have much more experience in EBLIP, but am new to assessment. These are my preliminary thoughts on the two, and how they might overlap, or fit together, or contradict one another.Trying to make sense of of EBLIP in my job as Assessment Librarian. Reserve the right to change my mind on all of this!Make sure to stop and give an outline of the talk:My understanding of assessment work and Why I should assess.An overview of the EBLIP movement and how it benefits librarianshipHow I have come to think about these two things: What does the EBL movement/framework have to offer assessment? And vice versaSome examples of integrating eblip (research evidence) into practiceSome examples of doing assessment (using local evidence) in practice
- Pam Ryan’s article on the two solitudes: “Beginning in 2005, individuals from one group have been showing up at the others’ conferences and events to discuss their methods, frameworks and processes. Are these separate movements within librarianship forming theoretical bridges? Is some sort of merger, fusion or takeover in the future?” 7 years later, I’m going to try to answer this question.
- Question: How many of you do assessment as part of your work, formally or informally?]
- in the Association of Research Libraries’ SPEC Kit 303: Library Assessment, Stephanie Wright and Lynda White state that “to assess, in general, is to determine the importance, size, or value of; to evaluate. Library staff assess operations by collecting, interpreting, and using data to make decisions and to improve customer service. They study internal processes, levels and quality of service, and library impact on institutional goals.”
- The definition of academic library assessment by Martha Kyrillidou and Pam Ryan and on the ARL supported library assessment blog.
- The goal of assessment is to inform decision-making for making changes/improvements (i.e., quality improvement – meeting the needs of users OR to demonstrate value/impact. “In assessment work, evidence can only be local. The nature of assessment work is that it circles around what users are experiencing, so the appropriate study design is frequently a qualitative, user‐engaging method.” Pam Ryan
- But what about what you can’t (or don’t) assess? Obstacles to assessment: time, data, resources (funding, support), experience, skills (from Kouf & Crumley, 2006)
- So what is evidence?In the cases where one is not doing assessment (for whatever reason), one can use evidence: credible, transferable findings from research to inform DM.
- Questions:A) How many of you are familiar with the term “EBLIP…”? Have heard it, know what it means
- Purpose of EBL: to create a framework or approach in which decision making can take place.
- 5 steps of EBLIP (the acronym for evidence-based library and information practice)Explain the steps
- But what does it mean to use research evidence?
- Changed the format to a linear process, just for illustrative purposes.
- I find that the steps have overlaps with other frameworks or processes that librarians are familiar with:
- IL standards – these are from ACRLThe Standards have a logical hierarchyEach standard is divided into several performance indicators.Standard OneThe information literate student determines the nature and extent of the information needed.(go on to explain the first 4 standards)
- Talk about assessment steps (research steps)I would think that EBL should be second nature to librarians already, it is similar to the IL process, which we advocate. Similarly, assessment is research, typically on a local level. Steps in assessment/research process are extensions of EBL/IL steps from a knowledge creator’s (rather than knowledge user’s) perspective.
- The eblip framework says that we should acquire published findings, and assessment research instructs us to gather local data to create findings for our setting.EBL is knowledge use, Assessment is knowledge creation (for use). Each is informed by the other. 2 pieces of a puzzle?Ideal is both together – “published” evidence that is credible and transferable + local evidence (data + interpretation = findings) = recommendations for action/changes or APPLICATION
- Give brief overview of datesFrom Pam Ryan: “The EBL and library assessment movements grew up separately but at approximately the same time, with EBL first appearing in the literature in 1995 in an editorial by Margaret Haines and the initial calls for a “culture of assessment” in academic libraries appeared in articles by Stoffle in 1996 and Lakos in 1999.”Also mention timing of LibQUAL+ development by Cook et al!
- Focus over last 15 years has been on scientific research, and coming out of the sciences/health model, emphasis was placed on study designs from clinical epidemiologist’s point of view – positivistic and therefore quantitative designs which favours controlled studies. These study designs are useful for assessing the effectiveness or success of specific interventions, but much research in LIS is not about that.Current state of EBL framework/movementDenise’s article on practice-based evidence, and Given’s article on qual research emphasize that not only different types of research, but not only “research” should be considered evidence for EBL. This evolution of the definition of EBL creates space for the type of evidence traditionally used for assessment – whether it be “small scale” evaluation or larger scale assessment of an entire department or library. EBL, then, includes assessment
- From Denise’s 2010 paper on practice-based evidence:Getting back to the definition of evidence:While research evidence is of high importance to our profession and knowledge, LIS practitioners need to first of all consider local evidence. Local evidence is found in our working environment and is specific to the context in which we carry out our work. It includes such things as our experience with patrons in particular contexts, and what we observe to work in our interactions, assessment of programs, feedback from our users, project evaluations, and accumulated experiences over the course of careers. These things are not easily shared and often do not find a place in publications because they are too local. But data that comes from a local context is in fact often the most important evidence source that a LIS professional can consult because it gives us information that is directly applicable to, and about our users. For example, usage stats on e-journals, feedback and comments about our services, usability testing on our website, titles from our interlibrary loan requests; these are just a few examples of local evidence that is invaluable to our decision making. This local data doesn‟t often mean much to others outside of our organization, but it is of utmost importance to our situated knowledge. The trick is to figure out what local information to systematically collect, and how to use it.
- Again from Denise:“They are all forms of evidence that come much more naturally to most LIS practitioners than research evidence.”
- Bringing evidence source together
- Includes needs assessment,which determines they what (and how much/relative importance) of the needs of users – and different user groups. Librarians are skilled at one type of user needs assessment – the reference interview. The reference interaction can be considered a micro-example of assessment: assessing needs, providing services/collections, assessing that needs have been met and the user is satisfied/more than satisfied. Librarian gains experience and knowledge which then feeds into the next transaction, and the cycle continues.[Follow up question: Does anyone want to revise their answer on whether or not they do assessment in their job?]
- Seating sweeps method – part of spacial analysis methods
- Readscale.orgThere is also the LEAD scale for library effort assessment data. It can be used for any library function, not just reference.
- In an article published in 2009, Megan Oakleaf describes the seven stages of the In- formation Literacy Instruction Assessment Cycle (ILIAC) and uses an assignment as an example to show how ILIAC provides a process for improving librarian teaching skills and student information literacy skills. The seven stages are: reviewing learning goals, identifying learning outcomes, creating learning activities, enacting learning activities, gathering data to check learning, interpreting data, and enacting decisions.
- Getting evidence into practiceGetting practice into ebl – Denise’s workGetting assessment into ebl –Pam Ryan, and now me.
- A revised process:Articulate – come to an understanding of the problem and articulate it.Assemble – assemble evidence from multiple sources that are most appropriate to the problem at hand.Assess(Appraise)– place the evidence against all components of the wider overarching problem. Assess the evidence for its quantity and quality.Agree – determine the best way forward and if working with a group, try to achieve consensus based on the evidence and organisational goals.Adapt –revisit goals and needs. Reflect on the success of the implementation. This cycle works for assessment as well.
- But instead of thinking of it as two types of work:One is assessment and brings local evidence (“Creation”)And one is EBLIP and makes use of published evidence (“use”)…..
- Both these types of evidence, and really, a broader definition of evidence to include professional knowledge
- When I read this definition, I think it means that we need research, analysis of local data, and facts/planning which comes from professional knowledge.So a culture of assessment is one that Is evidence-based.