Overview of student source use in psychology assignments
1. Dr. Nancy Evans Weaver
Principal Lecturer, Psychology
Estelle Barnard
Subject Librarian, Psychology
Open Polytechnic of New Zealand
2. Overview of Presentation
Introduction: Background and rationale
Method: Participants, Materials, Procedure
Results: Descriptive statistics, Statistical testing
Discussion: Summary of findings, critique, future
plans
3. Introduction
Estelle’s question:
What library sources do psychology students use?
Expanded version:
What sources from the scholarly literature do tertiary-
level students use in completing their assignments?
Collaborative nature of our research
Nancy: Psychology lecturer, teaching in one of our
large introductory psychology courses
Estelle: Psychology librarian, helping lecturers to
develop resources for students in this course.
4. Our Goals
Use our data to advise faculty.
Do faculty expectations about type and frequency of use
match student use patterns?
How can faculty improve instructions about source use
in scholarly writing?
Use our data to advise librarians.
Are current library resources used by students?
How can library programmes better fit needs?
5. Our Data
We collected data on:
Sources used
Textbook?
Instructor-provided sources?
Library-provided sources?
Student characteristics
Previous and current study
Assignment characteristics
Submitted online or in print?
Mark received?
6. Method
Participants
Students enrolled in 73195 general & applied
psychology, trimester 1 2010
124 students
Previous and current study
55% had NCEA levels 2 or 3
53% had previously enrolled at OP for a course(s)
67% studying only this course this trimester
7. This course
One of a pair of Intro Psych courses at the Open
Polytechnic
ODL (online and distance learning)
Assignment 1: Set up, run, and report on a
psychological experiment on memory
Specifically instructed in APA citing and referencing
(counts for 5% of mark)
8. Materials
Collected the References page from Assignment 1 for
all 124 students
Coded to remove identity
Coded to record type of submission (in print or online)
9. Procedure
Stratified random sample (n=12) of entire
assignments pulled and checked to see how
accurately References page captured sources cited.
32/36 sources on References page were cited (as they
should be) within the assignment: Accuracy = 88.89%
32/33 cited sources were on References page (as they
should be): Accuracy = 96.97%
Categorized and counted sources used
10. Results
Two types of data analysis
1. Descriptive statistics: Categories, with central
tendencies, ranges, frequencies
2. Statistical testing: Chi square tests of 3 hypotheses
11. Results: Descriptive Statistics
Total number of sources used
Mean = 3.62 sources (none required)
Range = 0 – 11
Median and Mode = 4
13. Results: Descriptive Statistics
Library sources
Subject Guide (online list of 73 library-held materials
specifically for this course and this assignment)
Frequency: 6%
Range: 0-2
Recommended and Additional Resources (online list of
17 library-held materials useful for psychology in
general)
Frequency: 2%
Range: 0-4
14. Results: Descriptive Statistics
Other sources
Not textbook, not instructor-recommended, not library-
provided
Found by student (?)
Frequency: 21%
Range: 0-7
15. Results: Descriptive Statistics
Type of submission
Students choose
In this assignment, 80% submitted online and 20%
submitted in print.
Mark achieved
Mean = 64.47
Mode = 75
Median = 67
Range: 18 to 91
16. Results: Statistical Testing
Just beginning – many more to run.
Based on pilot study (2009) and experience in this
course, we tested two general effects:
Hypothesis 1: There would be an effect of type of
submission.
Hypothesis 2 : Some source variables would affect
mark achieved.
17. Results: Statistical Testing
Hypothesis 1:
There would be an effect of type of submission.
Specific test:
Use of journal articles would differ between online and print
submissions.
Result:
When we grouped the articles used (no articles used, 1 article
used, 2 articles used, all 3 articles used), there is the suggestion
of an effect. Online submissions may have used more of these
articles than did print submissions (Chi square = 7.44, df = 3, p = .059).
18. Results: Statistical Testing
Hypothesis 2:
Some source variables would affect mark achieved.
Specific Test # 1:
The number of sources used would be related to the
mark achieved.
Result:
Grouping number of sources (0-1, 2-3, 4 or
more), there is a weak effect on mark. Assignments
using more sources got higher marks (Chi square =
12.59, df = 6, p = .05).
19. Results: Statistical Testing
Hypothesis 2:
Some source variables would affect mark achieved.
Specific Test # 2:
The use of instructor-recommended articles would be
related to the mark achieved.
Result:
Grouping article use (no article used, 1 used, 2 used, all
3 used) and grouping marks into letter grade categories
(0-49, 50-59, 60-74, 75-100), there is a significant effect
on mark. Assignments using more of these articles got
higher marks (Chi square = 24.78, df = 9, p < .01).
20. Results: Summary
We categorized source types, collected descriptive statistics on
all of them, and ran some statistical tests.
On average, students used 3 or 4 sources in this assignment.
The most frequently used sources were the instructor-provided
set of three journal articles, the textbook, and the instructor-
provided background.
Library sources were infrequently used.
Statistical testing suggests some effects of type of submission
and two variables that affect mark.
21. Discussion
Can we measure the types and frequencies of sources
used by tertiary-level students beginning their study
of psychology?
Yes.
Our results relate well to a body of literature on the use
of citation analysis. Journal articles are often used
heavily in student writing, and advice from faculty
affects what sources students use (e.g., Krause, 2002).
We found both these effects among ODL students at
the Open Polytechnic.
22. Surprises
More use of sources than we expected, given that none
(other than their own data) are required:
average of 3 – 4 sources used, with one student using 11.
As we found in the pilot study, assignments submitted
online may differ in some ways from those submitted in
print.
Very low use of library sources, even those specifically
geared to this assignment in this course.
23. Limitations to our work
Very limited sample:
Students in one course, in one trimester, in one discipline,
at one institution (which is ODL).
Limitations involving participants:
We assume competency in citing and referencing.
We assume honesty in citing and referencing.
Limitations involving statistics:
We have only begun testing.
Some categories have low numbers, which can affect results.
24. Future directions
Finish stats testing.
Analyse data from Assignment 2.
Collect and analyse data from assignments in a Level 6
course for which 73195 is a prerequisite.
25. Thanks
Our colleagues for help with data collection and
analysis.
Our students for their hard work in learning how to
run an experiment and write a scholarly report.
You, our audience today. Comments and suggestions:
Nancy.Weaver@openpolytechnic.ac.nz