This document summarizes existing research on evaluating online information literacy tutorials (OILTs). It finds that the most common evaluation methods are usability testing and pre- and post-tests. It also finds that each study evaluates a specific OILT rather than using standardized evaluation criteria. The document then proposes categorizing OILTs in different ways, such as modular vs. non-modular and use of text vs. video, before developing comprehensive evaluation standards. It concludes that OILTs should be evaluated according to these categories to ensure elements like appropriate length, interactive components, and accuracy of resources.

1. Identifying Evaluating
Standards for Online
Information Literacy
Tutorials (OILTs):
A Review of Existing OILTs
Evaluation Studies
Hang Dong
Presented in 3th September 2013
Summer School, University of Sheffield
This work is licensed under a Creative
Commons Attribution-NonCommercial-
ShareAlike 4.0 International License.

2. What is Information Literacy?
 Information literacy is a set of abilities requiring
individuals to "recognize when information is needed and
have the ability to locate, evaluate, and use effectively
the needed information." (American Library Association,
1989)
 “knowing when and why you need information, where to
find it, and how to evaluate, use and communicate it in
an ethical manner.” (CILIP, 2012).

3. Information Literacy abilities (ALA)
 Determine the extent of information needed
 Access the needed information effectively and efficiently
 Evaluate information and its sources critically
 Incorporate selected information into one’s knowledge
base
 Use information effectively to accomplish a specific
purpose
 Understand the economic, legal, and social issues
surrounding the use of information, and access and use
information ethically and legally

4.  Information literacy is “more than just the ability to acquire
a set of skills or an understanding of their application or
execution”. Information Literacy is a “complex and dynamic
practice that is driven by context”. (Lloyd, 2010 : 24-28)
 Information literacy appears various in different context,
Educational-----------------------------------textual, computer literacy
Workplace------------------------------------textual, computer literacy
Visual, such as art or media studies------visual literacy
Oral practice in society---------------------oral information literacy

5. What are Online Information Literacy
Tutorials(OILTs)?
 OILTs, Online Information Literacy Tutorials:
Information literacy tutorials offered via Internet, also called
web-based information literacy tutorials.
 It has been very popular since the last several years of 20C.
 OILTs have a clear positive impact on students’ academic
skills. (McClure et al., 2011; Donaldson, 2001; Bury and
Oud, 2005)
 OILTs are at least as effective as face-to-face or in-person
information literacy instructions.(Orme, 2004; Silver &
Nickel, 2005; McClure et al., 2011)

6. Example:
University
of Sheffield
Library
Website

9. University
of Illinois
at Urbana-
Champaign

10. Research question and method
 How do people evaluate these OILTs in academic
libraries?
 Is there a standard for all OILTs evaluation?
 Method:
 Documentary analysis

11. Existing research about OILTs (1)
Reference Research Topic OILT name Period of Research Evaluation
Method
McClure et
al., 2011
OILT and student
writing
The Search for
the Skunk Ape
Fall 2009-Spring
2010
citation, textual
analysis of
students’ work
Yelinek et al.,
2010
LibGuides in an
OILT
General
Library Research
Tutorial
August 2009 Not a rigorous
study
Bowles-Terry
et al., 2010
Best practices for
online video
tutorials
The University of
Illinois OILT
Before December
2010
Usability testing
Bury and Oud,
2005
Usability testing of
an OILT
Wilfrid Laurier
University
Before January
2005
Usability testing

12. Existing research about OILTs (2)
Reference Research Topic OILT name Period of
Research
Evaluation Method
Noe and
Bishop, 2005
OILT assessment Auburn University
Library’s Tiger IL
Tutorial (TILT)
Summer
2002-Fall
2003
IL pre-test and
post-test and
students'
perception survey
Orme, 2004 OILT and
Information-
Seeking Ability
Texas Information
Literacy Tutorial
Fall 2001-
Before May
2004
IL survey and
comparison
between 4
instructions
Donaldson,
2000
OILT design Library Research
Success
Before
Winter 2000
Students'
perception survey

13. Documentary Analysis
 (1) From selected papers, all of the studies concern an
individual OILT designed by an academic library.
 (2) The usability testing and pre/post-test comparison are
the most common methods for OILTs evaluation.
 (3) Each research summarises a set of standards
for an individual OILT or a type of OILTs.
 (4) It is reasonable and important to categorise
these OILTs in different ways, before forming a
comprehensive standard.

14. Categorises of OILTs
 1 Modular tutorials and non-modular tutorials
Initial purpose Content Length Others
Modular
tutorials
designed for
class instructions
Both theoretical and practical
knowledge
supposed to
be about 20-
30 min
Interactive
elements and
a script
Non-
modular
tutorials
not designed for
class instructions
Clearly emphasised on
practical knowledge, and de-
emphasised on theoretical
knowledge
Sometimes
much shorter

15.  Texas Information Literacy Tutorial (TILT), integrated “into
first-year college courses” (Kasowitz-Scheer and Pasqualoni,
2002 : 3)

16. Non-
modular
tutorial
University
of Illinois
at Urbana-
Champaign

17.  2 Different expressions of tutorials
Text tutorials-------indispensable.
Video tutorials,
often for specific and discrete tasks,
e.g. cite use Endnote
e.g. find a paper from Library website
Clear explanations of concepts
Concise explanations of concepts
Well-defined concepts
Comprehensive examples
Have captions
Have a simple look
Have a reasonable length
Have a reasonable pace (slow)
Have a friendly layout
Easy to be found when needed

18.  3 Technologies used in tutorials
LibGuide or subject guide tutorials
“a Web 2.0-based content management and information dissemination
system” (Mooney, 2012 : 1)
“3,000,000 guides by 60,000 librarians” (Springshare, 2013)
Easy-to-use, standardised tool, including widgets and RSS feeds,
Mobile-friendly,
but it needs frequent maintaining for “accuracy and currency”.
(Emanuel, 2012 : 11; Leibiger and Aldrich, 2013 : 431)
E.g. The “General library Research Tutorial”
Other technologies
Blog tutorials, Web games, virtual 3D tutorials

19. OILT name Period of Research Modular Text Video Subject Guide
The Search for the Skunk Ape Fall 2009-Spring 2010 1 1 1 0
General Library Research Tutorial August 2009* 1 1 0 1
The University of Illinois OILT Before December
2010
0 1 1 0
Wilfrid Laurier University Library
OILT
Before January 2005 1 1 0 0
Auburn University Library’s Tiger
Information Literacy Tutorial
Summer 2002-Fall
2003
1 1 0 0
Texas Information Literacy Tutorial Fall 2001-Before May
2004
1 1 0 0
Library Research Success Before Winter 2000 1 1 0 0

20. Conclusion
It is essential for librarians to evaluate different OILTs
according to their categories.
For modular
tutorials
Separated to different modules
Appropriate length for each modules
Have interactive elements (quizzes)
Be companied with a script
Have theoretical and practical
contents
For
non-modular
tutorials
Highlight practical contents
Entitled with words like "How to"
Relatively short learning time for
each unit

21. Have text
webpages
Clear explanations of concepts
Concise explanations of concepts
Well-defined concepts
Comprehensive examples
Have videos Have captions
Have a simple look
Have a reasonable length
Have a reasonable pace (slow)
Have a friendly layout
Easy to be found when needed
Have subject
guides
Ensure resources' currency
Ensure resources' accuracy

22. Limitations of the study
 The samples of papers need to be more recent in general.
 The numbers of samples need to be expanded.

23. Main References
 American Library Association. Information Literacy Competency Standards
for Higher Education[Online]
http://www.ala.org/acrl/standards/informationliteracycompetency
 CILIP. Information literacy – Definition[Online]
http://www.cilip.org.uk/cilip/advocacy-campaigns-awards/advocacy-
campaigns/information-literacy/information-literacy
 LLOYD, A. (2010). Information literacy landscapes: information literacy in
education, workplace and everyday contexts. Oxford, UK, Chandos Pub.
 McClure, R., Cooke, R. and Carlin, A. (2011). The search for the Skunk Ape:
studying the impact of an online information literacy tutorial on student
writing. Journal of Information Literacy, 5(2), 26-45.
 Yelinek, K., Neyer, L., Bressler, D., Coffta, M., Magolis, D. (2010). Using
LibGuides for an information literacy tutorial Tutorial 2.0. College &
Research Libraries News, 71(7), 352-355.

24.  Bowles-Terry, M., Hensley, M., & Hinchliffe, L. (2010). Best practices
for online video tutorials in academic libraries. Communications in
Information Literacy, 4(1), 17-28.
 Bury, S. and Oud, J. (2005). Usability testing of an online information
literacy tutorial. Reference services review, 33(1), 54-65.
 Noe, N. W. and Bishop, B. A. (2005). Assessing Auburn University
Library's Tiger Information Literacy Tutorial (TILT). Reference services
review, 33(2), 173-187.
 Orme, W. A. (2004). A study of the residual impact of the Texas
Information Literacy Tutorial on the information-seeking ability of first
year college students. College & Research Libraries, 65(3), 205-215.
 Donaldson, K. A. (2001). Library research success: designing an online
tutorial to teach information literacy skills to first-year students. The
Internet and Higher Education, 2(4), 237-251.

25. Thank you for your
attention