Upstairs-downstairs: Working with a campus assessment coordinator and other allies for effective information literacy assessment

UPSTAIRS-DOWNSTAIRS:
Working with a campus assessment coordinator and
other allies for effective information literacy assessment

Amy Hofer, Research Instruction Librarian
Margot Hanson, Web Services Librarian
Golden Gate University
CARL Conference, Sacramento CA, April 9, 2010

Monday, April 12, 2010

ASSESSMENT CYCLE OR CIRCULAR FILE?


GOLDEN
GATE
UNIVERSITY
about us!


RESEARCH INSTRUCTION

ACADEMIC CHAMPIONS

PLUS
Preparation in Language & University Studies


PROGRAM EVALUATION
Study your program to ﬁnd out whether it’s meeting your goals


LEARNING ASSESSMENT
Study whether students gained knowledge, skills, and abilities as
a result of your instruction


THE OLD
SURVEY
It’ll be bigger in a minute!


1) Will this library instruction
session help you with your
research assignment for this class?

Yes Maybe No


2) What did you ﬁnd valuable about
the session? (mark all that apply)
Learning about library services
Learning about developing keywords
Learning how to select relevant resources
Learning how to search databases
Finding out about online research guides
Receiving course-speciﬁc handouts
Learning about when to cite sources
Other, please specify:

3)What information received
today is still unclear to you?


4) What would you have liked to
learn that was not covered in this
session?


THE OLD
SURVEY
‣ Timing - right after
instruction
‣ How can we use
this information?


THE LITERATURE

OFF TO THE RACES

UPSTAIRS-
DOWNSTAIRS
Program Evaluation Group
and other allies


LESSONS FROM LISA
Identify Study Objectives

‣ For what general purpose is the evaluation being
conducted?

‣ What are the speciﬁc goals of the study?

‣ Who is the audience for research ﬁndings?

‣ What kinds of information are needed to make the
decisions you need to make?


LESSONS FROM LISA
What data collection method is appropriate?
‣ Test/Quiz ‣ Focus group interviews

‣ Systematic examination of ‣ Field observations
work samples
‣ Secondary data analysis
‣ Survey
‣ Mixed methods
‣ Individual interviews approaches


OUR AREAS OF INQUIRY
‣ A. Do students identify appropriate sources for their
research need?

‣ B. Do students ask a great question that ﬁts the scope of the
assignment and requires research and analysis to answer?

‣ C. Do students demonstrate academic integrity by correctly
citing sources?

‣ D. Do students use the library’s resources and services as
part of the research/writing process?

‣ E. Qualitative info about students

FOR EXAMPLE:

If you have just found 5 sources for your research paper,
what is the best next step to take?

a. Scan (pre-read) a little bit of what you already have to ﬁnd out whether
you need more information

b. Brainstorm new topics in case you want to change your mind

c. Search for 5-10 more articles

d. Print everything out

e. I don’t know


FOR EXAMPLE:
Which of the following topics is a great idea for a
research paper?

a. How has the Cash for Clunkers stimulus program affected the economic
well-being of single mothers who lost their homes in Hurricane Katrina?

b. Cancer

c. Is communication the key to good management skills?

d. Is it ethical to target advertising to very young children in order to build
long-term brand loyalty?

e. I don’t know

FOR EXAMPLE:
Why should you cite your sources?

a. If you don’t cite, you’re stealing somebody else’s words or ideas

b. So that your professor can ﬁnd the source of the information you
used

c. Because it is the expectation of the academic community that scholars
build on the ideas of others

d. All of the above

e. I don’t know


FOR EXAMPLE:
What is the most important difference between searching
for information on Google vs. in the library’s databases?

a. Google always gives you the full text of the resource

b. Library databases give you access to in-depth information not available on
Google

c. Google is better because you will always get more hits

d. You don’t need to evaluate the information you ﬁnd in library databases
because it is always scholarly

e. I don’t know


Didn't Get It Got It Got It & Ran With It!

•Uses a range of formats,
including reports, articles and
•Uses only one format •Uses a range of formats, websites
including reports, articles and
Appropriate •Not relevant or authoritative websites
•Includes resources only available
Sources •Heavy reliance on one or two •Uses only resources presented
from the library, but went beyond

You don’t have to read this -
sources resources presented in class
in class
•Sources are relevant and
substantive

it’s online!
•Topic too narrow or too broad •Research question is of
•Uses critical thinking and
creative approach to topic, which
is of appropriate scope
for assignment appropriate scope for assignment
Topic
• Uses report format, as a simple •Includes original analysis •Research question requires
Development original analysis
statement of facts, missing original •Question can be answered with
analysis resources presented in class •Question requires additional
sources beyond what was
presented in class

•Errors in APA formatting •APA correctly and consistently
•Incorrect use of paraphrasing, •Consistent APA formatting in- used for in-text and reference list
quotations, or summarizing
Citation •Quotes are dumped into paper text and reference list citations
and not incorporated into analysis • •Citations are distributed evenly
Originality report reveals less
than 10% exact match throughout paper, contribute to
•Originality report reveals analysis, and support conclusions
problems with exact matches


OUR RUBRIC

Didn’t get it
Got it
Got it & ran with it!


FOR EXAMPLE:
Did the student use appropriate sources?

‣ Didn’t get it: Uses only one format; not relevant or
authoritative; heavy reliance on one or two sources

‣ Got it: Uses a range of formats, including reports, articles
and websites; uses only resources presented in class

‣ Got it & ran with it!: Uses a range of formats, including
reports, articles and websites; includes resources only available
from the library, but went beyond resources presented in
class; sources are relevant and substantive


PRELIMINARY FINDINGS


OMG, data!


Most Improved Players
70%
66% 63%

54% 60% 56%
49%
42%
35%
28%
21%
14%
7%
0%

PLUS pretest PLUS posttest Undergrad


More Detail
40%

35%

30%

25%

20%

15%

10%

5%

0%
0 1 2 3 4 5 6 7 8 9 10



CATEGORICALLY SPEAKING


Do students identify appropriate sources for their
research need?
60%
60%
53% 55%
45%

30%

15%

0%



Do students ask a great question that ﬁts the scope of the
assignment and requires research and analysis to answer?
63%
63%
57%
53% 47%

32%

16%

0%



Do students demonstrate academic integrity by
correctly citing sources?
80%

72%
68% 60%

48%
40%

20%

0%

PLUS pretest PLUS posttest UG


Do students use the library’s resources and services as part of
the research/writing process?
90%
90%

70% 68%

57%
45%

23%

0%

PLUS pretest PLUS posttest UG

What the student papers told us
11
2 3 10
9
6 8
7
6
7 5
6 4
3
5 2
1
2 2
Appropriate sources Topic development Citation

Got it & ran with it!
Got it
Didn’t get it

Do students know what they know?
20

15

10

5

0
0 1 2 3 4 5 6 7 8 9 10

My skills are excellent
My skills are good
My skills are fair
My skills are poor
I don’t know

ERROR
ANALYSIS


Image sources:
http://www.flickr.com/photos/seattlemunicipalarchives/2680241972/
http://www.flickr.com/photos/telmo32/2967379555/
http://www.un.org/esa/socdev/unpfii/media/images/UN-LOGO
%20copy.jpg
http://www.flickr.com/photos/princessashley/1546575032/
http://www.wordle.net
http://www.flickr.com/photos/kevinomara/415961040/
http://www.flickr.com/photos/curtisnewton/4089464649/
http://www.flickr.com/photos/gadl/320300354/
http://www.flickr.com/photos/michaelcr/1425130599/
http://www.flickr.com/photos/mcaven/4319012152/
http://www.flickr.com/photos/daikrieg/1294053038/
http://www.uscibooks.com/Taylorposter.jpg
http://www.streetsigngenerator.com


Thank you!
Any questions?

Amy Hofer Margot Hanson
ahofer@ggu.edu mhanson@ggu.edu

DISCUSSION STARTERS
1. A question from the ILI list: How do you evaluate evidence of critical
thinking?

2. If you feel that your current assessment efforts are ineffective, can your
group identify institutional or methodological stumbling blocks that you
could work on?

3.Do you have a spectacularly successful or unsuccessful assessment
experience to share with your group?

4.Are you currently doing assessment that is: program evaluation; learning
assessment; satisfaction surveys? Is it working for you?

5.Do you currently have partners in assessment at your institution? Are there
individuals or groups you would like to collaborate with?

Upstairs-downstairs: Working with a campus assessment coordinator and other allies for effective information literacy assessment

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (7)

Semelhante a Upstairs-downstairs: Working with a campus assessment coordinator and other allies for effective information literacy assessment

Semelhante a Upstairs-downstairs: Working with a campus assessment coordinator and other allies for effective information literacy assessment (20)

Mais de Margot

Mais de Margot (13)

Último

Último (20)

Upstairs-downstairs: Working with a campus assessment coordinator and other allies for effective information literacy assessment

Notas do Editor