1. From Community Source to Open Data:
Leveraging the Sakai Community
to Make Better Decisions
Steven Lonn
Research Fellow, University of Michigan
Mathieu Plourde
Project Leader, LMS, University of Delaware
Stephanie D. Teasley
Associate Research Professor, University of Michigan
USE Lab • Digital Media Commons • University Library • University of Michigan
IT Client Support & Services • University of Delaware
1

2. Session Overview
• The “average” IT survey
• The Sakai community & building a
shared survey instrument
• High-level results and interesting
differences
• Using survey data to promote local
innovations
2 USE Lab
University of Michigan
http://umich.edu/~uselab
2

3. Survey Respondents at
the University of Delaware
• Topics:
- Use of technology for academic purposes
- Special focus on the Sakai LMS
• Respondents:
- 51 Faculty
- 361 Students
3 USE Lab
University of Michigan
http://umich.edu/~uselab
3

4. Rating the Value of
Information Technologies
Q. The following information technologies are
Strongly valuable for my course-related activities…
5
Agree 4.7
4.4
4
3.6 3.6
3.0
3 2.7
2
Strongly
1
Disagree Social
Sakai or Handheld
Networking
Other LMS Devices
Websites
4
Instructor Student USE Lab
University of Michigan
http://umich.edu/~uselab
4

5. Observations
• Instructors and students highly
value the use of a LMS.
• Students value the use of social
media and handheld devices
for learning more than instructors.
5 USE Lab
University of Michigan
http://umich.edu/~uselab
5

6. Observations
is.
all do th
e
W and studentsusers
• Instructors ey our highly
W the
value rv of a LMS. alyze
e su use , an
od ea th)
(t m ake
a,use of social
the dat
• Students value the pu ta
media sions, and devices
and handheld
eci more thancinstructors.
d learning to a tio
for
n.
pla n in
5 USE Lab
University of Michigan
http://umich.edu/~uselab
5

7. What i
Observations
f wthcs.
e i ould
comall do
e pare ourusers
W and students ow
• Instructors ey our highly
inutiv u
s r t tio n
W e s use of a,LMS. aithe
value the nn lyz
aw
od our h )
eat peersake
(t , m?
th ed ata use of social
ta
• Students value the pu
and devices
media shats,
W ion c
and handheld
eci moreoulcinstructors.
d learning to a d io
for than t
n.
we lea
pla n in rn?
5 USE Lab
University of Michigan
http://umich.edu/~uselab
5

8. The Sakai Community
• Began as a research project:
Sharing code on a Collaboration and
Learning Environment (CLE)
• Now: Over 200 institutions
- Many contribute software code
under an open, community license
6 USE Lab
University of Michigan
http://umich.edu/~uselab
6

9. The Sakai Community
• Non-technical professionals:
- Leveraging the network of
Sakai institutions
- Understand how
technology is used to
support teaching & learning
7 USE Lab
University of Michigan
http://umich.edu/~uselab
7

10. From Local to Global
8 USE Lab
University of Michigan
http://umich.edu/~uselab
8

11. From Local to Global
2011
2003
8 USE Lab
University of Michigan
http://umich.edu/~uselab
8

12. Co-Constructing a Survey
• The Sakai Multi-Institutional Survey Initiative (MISI)
• Goal: Gauge user attitudes,
perceptions, and use.
• Method: Locally-administered online
surveys, all using a “core” set of
survey items.
9 USE Lab
University of Michigan
http://umich.edu/~uselab
9

13. Sausage-Making: Survey-Style
• Shared wiki
• Conference calls
• Email
10 USE Lab
University of Michigan
http://umich.edu/~uselab
10

14. Survey Instrument
• General Information Technology
(IT) use / preferences in courses
11 USE Lab
University of Michigan
http://umich.edu/~uselab
11

15. Survey Instrument
Using information technology
in my course(s) is valuable for...
(5-point Likert scale: Strongly Disagree - Strongly Agree)
• General Information Technology
• Improving my teaching
(IT) use / preferences in courses
• Improving my students’ learning
• Saving me time (efficiency)
• Accessing material any time, from any location (convenience)
• Managing my course activities
• Improving my students’ communication to me
• Improving my communication to my students
• Improving my students’ communication with each other
11 USE Lab
University of Michigan
http://umich.edu/~uselab
11

16. Survey Instrument
• General Information Technology
(IT) use / preferences in courses
• LMS activities and perceptions
11 USE Lab
University of Michigan
http://umich.edu/~uselab
11

17. Survey Instrument
In terms of presenting and / or sharing materials,
Sakai is valuable for...
(5-point Likert scale: Strongly Disagree - Strongly Agree)
•
• Posting a syllabus Information Technology
General
• Posting lecture outline or notes after the lecture
(IT) use / preferences in courses
• Posting sample exams & quizzes for learning purposes
•
Within Sakai, the followingperceptions
LMS activities and tools are valuable for
my course-related activities...
• Assignments
• Discussion
• Gradebook
11 USE Lab
University of Michigan
http://umich.edu/~uselab
11

18. Survey Instrument
• General Information Technology
(IT) use / preferences in courses
• LMS activities and perceptions
• Quantitative and qualitative items
11 USE Lab
University of Michigan
http://umich.edu/~uselab
11

19. Survey Instrument
Has•using Sakai affected how you use
General Information Technology
(IT) use / preferencesso, courses
in-class time? If in how?
• LMS activities and perceptions
• Quantitative and qualitative items
11 USE Lab
University of Michigan
http://umich.edu/~uselab
11

20. Data Collection & Analysis
• Surveys individually administered by each participating
institution
• Data then
uploaded to
Michigan's Sakai
implementation
12 USE Lab
University of Michigan
http://umich.edu/~uselab
12

21. Participants
• 3,431 Instructors
• 8,228 Students
13 USE Lab
University of Michigan
http://umich.edu/~uselab
13

22. Survey Items by Interaction Type
• Learner-Content (LC)
14 USE Lab
University of Michigan
http://umich.edu/~uselab
14

23. Survey Items by Interaction Type
• Learner-Content (LC)
• Learner-Instructor (LI)
14 USE Lab
University of Michigan
http://umich.edu/~uselab
14

24. Survey Items by Interaction Type
• Learner-Content (LC)
• Learner-Instructor (LI)
• Learner-Learner (LL)
14 USE Lab
University of Michigan
http://umich.edu/~uselab
14

25. Survey Items and
Interaction Types
Learner- Learner- Learner-
Content Instructor Learner
• Post / Access online readings • Send / Receive messages or • Students work together on a
and supplementary materials notifications task or assignment
• Post / Access lecture outline • Students turn in assignments • Students read / comment on
before lecture • Instructors return each others’ work
• Post / Access lecture outline assignments with comments • Students generate / share
after lecture and/or grade instructional materials
• Post / Access multimedia • Students ask questions • Instructors create / Students
materials before lecture are part of ad-hoc student
• Students ask questions after groups or teams
lecture
• Give / Take online exams and
quizzes
• Post / Access course or
assignment grades
15 USE Lab
University of Michigan
http://umich.edu/~uselab
15

26. Research Questions
• Do instructors and students value interaction
types differently?
• Do respondents in research and non-research
universities value interaction types differently?
• Does the number of courses for which one has
used an LMS influence how much one values specific
interaction types?
• Does how much one uses IT in courses influence
how much one values specific interaction types?
16 USE Lab
University of Michigan
http://umich.edu/~uselab
16

27. Independent
Variables
• Same independent variables used across all
three HLMs
Key Control
•Instructor / student •Expertise with computers
•Research / Non-research •Frequency of LMS visits
university •Value of LMS
•Number of courses one •Value of IT for improving
has used an LMS teaching & learning
•Preference for IT
17 USE Lab
University of Michigan
http://umich.edu/~uselab
17

28. 18
USE Lab
University of Michigan
http://umich.edu/~uselab
Non-Research
Use IT
# of Courses
Key Variables
Student
Intercept
Non-Research
Use IT
# of Courses
18
Student
Intercept
Non-Research
Use IT
# of Courses
Student
Intercept

29. 19
USE Lab
University of Michigan
http://umich.edu/~uselab
IT Improve Learning
Control Variables
IT Improve Teaching
Value LMS
Expertise
Frequently Visit
IT Improve Learning
IT Improve Teaching
Value LMS
19
Expertise
Frequently Visit
IT Improve Learning
IT Improve Teaching
Value LMS
Expertise
Frequently Visit

30. Instructor / Student
Differences
• Students value LC and LI
interactions more highly than
instructors
- Students value LI interactions
more than LC interactions
20 USE Lab
University of Michigan
http://umich.edu/~uselab
20

31. Research / Non-
Research
• Individuals within non-
research universities value
LL more than individuals
within research universities
- Largest single factor for
how much one values LL
interactions
21 USE Lab
University of Michigan
http://umich.edu/~uselab
21

32. Number of Courses
• Positively influences how much one values
LC interactions
• Negatively influences how much one
values LL interactions
22 USE Lab
University of Michigan
http://umich.edu/~uselab
22

33. Use of
Information Technologies
• Significant, positive
effect across all
models
- Largest effect for LL
interactions
compared to other
interaction types
23 USE Lab
University of Michigan
http://umich.edu/~uselab
23

34. Implications
• Research focusing on different
instructional uses and at different
levels of analysis will help shed light
on what is driving the differences
24 USE Lab
University of Michigan
http://umich.edu/~uselab
24

35. Bringing it Home:
Supporting Local Innovations
• Using Sakai since 2004
• Reference point for new
adopters
• New tools such as online
course evaluations,
textbooks, and increasing use
of LTI-powered tools
25 USE Lab
University of Michigan
http://umich.edu/~uselab
25

36. Bringing it Home:
Making Support Decisions
• Using Sakai since 2008
• Most Sakai tools used
out-of-the box
26 USE Lab
University of Michigan
http://umich.edu/~uselab
26

37. How do Instructors at
Research Institutions Value IT?
Q. The following information technologies are valuable
for my course-related activities…
5 4.7
4.5 4.5
4.3 4.1
4.1 4.1 4.1 4.0 4.0
3.6
4 3.3 3.4
3.2 3.1
2.9
3
1
0
Sakai or Other LMS
Online Media
Quiz / Test systems
Web-based file server
Personal website
Group calendar
Plagiarism-prevention tools
Clickers
Instructors-UD 27 USE Lab
University of Michigan
Instructors-Other http://umich.edu/~uselab
27

38. How do Instructors at
Research Institutions Value IT?
Q. The following information technologies are valuable
for my course-related activities…
5 4.7
4.5 4.5
4.3 4.1
4.1 4.1 4.1 4.0 4.0
3.6
4 3.3 3.4
3.2 3.1
2.9
3
1
Whoa,
0 what’s
Sakai or Other LMS
Online Media
Quiz / Test systems
Web-based file server
Personal website
Group calendar
Plagiarism-prevention tools
Clickers
up with
that?
Instructors-UD 27 USE Lab
University of Michigan
Instructors-Other http://umich.edu/~uselab
27

39. Possible Actions
28 USE Lab
University of Michigan
http://umich.edu/~uselab
28

40. Conclusion
• We all do surveys
• Value of the instrument-creating process
• Networking benefits
• Replicable for any purpose, in any community
29 USE Lab
University of Michigan
http://umich.edu/~uselab
29

41. Contact Us
Steven Lonn Mathieu Plourde
slonn@umich.edu mathieu@udel.edu
stevelonn mathplourde
http://umich.edu/~uselab http://www.udel.edu/udlms
USE Lab • Digital Media Commons • University Library • University of Michigan
IT Client Support & Services • University of Delaware
30