SLN SOLsummit 2009 presentation - by Peter Shea http://slnsolsummit2009.edublogs.org New Approaches in Online Learning Research This presentation will share new research towards a theory of online teaching and learning. This ongoing investigation employs novel research methods including quantitative content analysis, social network analysis, and cluster analysis to further our understanding of pedagogical, social, and cognitive processes essential for quality online education.

1. Peter Shea
SLN Senior Researcher
Educational Theory
and Practice & CCI
University at Albany
Community of Inquiry II:
New Research on the
Future of Online Learning
SLN SOLsummit Meeting
February, 2009

2. Overview
• New research towards a theory of online
teaching and learning.
• Research methods including
– quantitative content analysis, social network
analysis, and cluster analysis
• Goal: further our understanding of
pedagogical, social, and cognitive
processes essential for quality online
education.

3. Overview: Community of Inquiry
Framework

4. > 1000 papers refer to
CoI foundational studies

5. Community of Inquiry Framework
Cognitive Presence
Social Presence
The extent to which
The ability of
learners are able to
participants
construct and confirm
to identify with the
meaning through
community (e.g., course
sustained reflection
of study), communicate
and discourse in a
purposefully in a trusting
critical community of
environment, and
inquiry.
develop inter-personal
relationships by way of
projecting their
individual personalities.
Teaching Presence
The design, facilitation and direction of cognitive and social processes for the purpose of
realizing personally meaningful and educationally worthwhile learning outcomes

6. Elements,Categories &
Indicators
ELEMENTS CATEGORIES INDICATORS
(examples only)
Social Presence Open Communication Learning climate/r isk-free expression
Group Cohesion Group identity/ collaboration
Personal/Affective Self projection/e xpressing emotions
Cognitive Presence Triggering Event Sense of puzzlement
Exploration Information exchange
Integration Connecting ideas
Resolution Appl ying new ideas
Teaching Presence Design & Organization Setting curriculum & methods
Facilitating Discourse Shaping constructive exchange
Direct Instruction Clarifying, explaining, demonstrating

7. Questions
• Do instructor skills in teaching presence
foster students’ social presence?
• Do student experiences of teaching and
social presence “predict” their experience
of cognitive presence?

8. Teaching Presence Indicators
When the online instructor:
• Communicates course topics, goals, and due dates
• Provides clear instructions
• Helps students clarify their thinking
• Keeps students on task, engaged, and participating
• Encourages students to explore new ideas
• Focuses discussion on issues that aided student learning
• Establishes a sense of course community
• Provides explanations or demonstrations to help learners better
understand the content
• Gives helpful feedback
• Asks for feedback on how the course could be improved
It leads to…

9. Social Presence
Students:
• gain a sense of belonging in the course
• form distinct impressions of course participants
• find online communication an excellent medium for social
interaction
• are able to identify with the thoughts and feelings of other
students
• feel comfortable conversing online
• feel comfortable participating in discussions
• feel comfortable interacting with course participants
• feel comfortable disagreeing with others
• online discussions helped students develop a sense of
collaboration
• feel their points of view are acknowledged by others
Which in turn leads to…

10. Cognitive Presence
• Course activities pique curiosity
• Problems posed increase interest in course issues
• Students feel motivated to explore content related topics
• Students brainstorm & find relevant information to aid them in
resolving questions
• Online discussions help students appreciate different perspectives
• Combining new information helps students answer questions
• Learning activities help students create solutions
• Reflection on course content & discussions help students
understand fundamental concepts
• Students can describe ways to test & apply their new knowledge
• Students develop solutions to course problems that can be
applied in practice
• Students can apply knowledge created in their courses to work or
other non-class related activities

11. Structural Equation Modeling

12. .4 0 .3 8 .4 1 .1 8 .2 9 .2 7 .6 2 .6 7 .6 5
1
1 1 1 1 1 1 1 1
q22 q21 q20 q19 q18 q17 q16 q15 q14
.77 .78
.59
.58
.86
.85
.62
.91
.77
Social 1
1
q34
Presence .4 4
q1
.3 7
.52 1
1
q33
.75
(.4 .3 9
q2
.4 0
9)*
* .78
1
1
q32
.78 .4 0
q3
*
.4 3
2)*
.80 .78
.85
(.5
1
Cognitive
1
.52
q31 .2 7
.86
q4
.5 3 .75 Presence
.84
.69
)**
1
1
q30 .2 7
q5
(.47
.2 4 .87
.7
.49
2
1
1 .90
q29 .2 9
q6
.1 9
.75
Teaching
Presence
.88
1
.72 .81 .80 2
1
q28 .4 9
q7
.2 3 .90
.85 1
1
q27 .4 4
q8
.1 8
.7
7
.8
1
1
.89 .85
q26 .4 7
q9
.2 7
1
1
q25 .3 4
q10
.2 4
.74
1
1
q24 ..3 6
q11
.2 8
1
.02
1
q23 .4 8
.06(.04)*
q12
.00
.3 8
(.08
(.0
1)
)**
1
q13
.4 6
Age Academic
Gender
Level
.22**
.06*
.00

13. Effects
Table 2: Unstandardized Path Coefficient and Total Effects
Path Unstanderdized Standardized
Coefficient Coefficient
Direct Effects
Gender to Teaching Presence .06* .04*
Age to Teaching Presence .02** .08**
Academic Level to Teaching Presence .00 .01
Teaching Presence to Social Presence .52** .52**
Teaching Presence to Cognitive Presence .49** .47**
Social Presence to Cognitive Presence .52** .49**
Total Effects
Gender to Social Presence .03 .00
Gender to Cognitive Presence .05 .00
Age to Social Presence .01 .00
Age to Cognitive Presence .02 .00
Academic Level to Social Presence .00 .01
Academic Level to Cognitive Presence .00 .01
Teaching Presence to Cognitive Presence .77** .72**
Note. *p<.05, **p<.001

14. Weaknesses of Model
• Research methods limited/problematic
• Based primarily in discussion
• Limited to one form of presence at a time
• Other areas of courses not considered
• Model categories biased toward what can
be discovered in discussions

15. Solutions
• Need to look at other areas of courses
• Use multiple methods
– QCA, SNA, Cluster Analysis
• Goal is to code entire courses for
– TP SP CP simultaneously
• Revise and expand these categories
• Develop more comprehensive model
• Build theory w/ dialogue between methods

16. Current study
• The current study goes beyond prediction of
student reports of “presence”.
• Quantitative content analysis of two complete
online courses with…
• Low v Higher Instructor Presence
• Goal: find TP, SP, and CP
patterns/associations in discussions and
other courses areas.
• Determine if previous research results can be
verified, refined, or extended.

17. Re-examining Affective Indicators
for Measuring Social Presence in
the COI Framework

18. Three Categories of Social Presence
Indicators (Garrison et al. 2000)
Affective Expression Expression of emotions,
feelings, mood
Open Communications Responses that build and
(Interactive Responses) sustain relationships; tacit
expressions of support,
encouragement, and
acceptance
Group Cohesion Activities that build and
sustain a sense of group
commitment; use of
greetings, names, “us” or
“we,” social sharing

19. Courses and Coding
Process

20. Coding Process
• Two coders
• Coded random module for practice
• Met to negotiate
• Identified problems
• Established and documented guidelines
• Recoded original module
• Message = unit of analysis
• Coded for presence/absence of SP indicators:
AF, OC, CH or NC

21. Original Codes from Rourke et al. 1999. Swan et al. 2001 in blue
SP Categories Indicators Code Definition
Expressing emotions SP-AFI
Affective (AF) Conventional expressions of emotion
Use of humor SP-AF2 Teasing, cajoling, irony, understatements, sarcasm
Self-disclosure SP-AF3
Presents details of life outside of class, or expresses vulnerability
Use of unconventional expressions SP-AF4 Unconventional expressions of emotion. includes repetitious
to express emotion punctuation, conspicuous capitalization, emoticons
Expressing values SP-AF5 Expressing personal values, beliefs and attitudes
Continuing a thread SP-OC-1
Open Communication
Using reply feature of software, rather than starting a new thread
(OC)
Quoting from others' messages SP-OC-2 Using software features to quote others' entire message or cut and
passing selections of others' messages
Referring explicitly to others' SP-OC-3 Direct references to contents of others' posts
messages
Asking questions SP•OC4 Students ask questions of other students or the moderator
Complimenting. expressing SP-OC-5 Complimenting others or contents of others' messages
appreciation
Expressing agreement SP-OC-6 Expressing agreement with others or contents of others messages
Expressing Disagreement SP-OC7 Expresses disagreement with other or contents of others messages
SP-OC-8
Personal Advice Offering specific advice to classmates
Vocatives SP-CH-I
Group Cohesion (CH) Addressing or referring to the participants by name
SP-CH-2
Addresses or refers to the group
Addresses the group as we., us, our, group
using inclusive pronouns
Phatics, salutations & greetings Communication that serves a purely social function; greetings or
SP-CH-3
closures
Social Sharing SP-CH-4 Sharing information unrelated to the course
SP-CH5
Course Reflection Reflection on the course itself

22. Two Identical 15-week Courses
Instructor KS Instructor JY
Duration of
Approximately 2 weeks
Discussions
Number of
One discussion from Modules 1-5
Discussions Coded
Total Number of
490 454
Discussion Postings
Number of Students
19 20
At Start of the Term
Number of Students
Completing the Term 17 16
Minimum Number of
2 to 3 postings a week
Required Postings

23. Initial and Negotiated Inter-rater Reliability using Cohen’s Kappa Initial and Negotiated Holsti’s Coefficient of Reliability
Course A (KS) Course B (JY) Course A (KS) Course B (JY)
Initial k Negotiated Initial k Negotiated Initial Negotiated Initial Negotiated
k k
Module 1 0.91 0.982857 0.90 0.981
Module 1 075 0.95 0.75 0.97
Module 2 0.91 0.980695 0.91 0.99
Module 2 0.76 0.94 0.78 0.99
Module 3 0.89 0.977778 0.87 0.97
Module 3 0.70 0.93 0.66 0.91
Module 4 0.89 0.998273 0.88 0.98
Module 4 0.71 0.99 0.71 0.96
Module 5
Module 5 0.75 1 0.82 1 0.99 0.90411 0.93 1.00
Pre-negotiated Instructor Instructor
average IRR* for 5 KS JY
Discussions
With All SP .73 .74
Categories/Indicators
With AF Indicators .85 .92
Removed

24. Findings
•Overall increase in SP indicators in Course
KS
•Overall decrease in SP indicators in Course
JY
Avg
Avg
Number of Individual Student Indicators per # of
Number of Individual Student Indicators per # of indicator
indicator per
Module students
Module students per student
student
AF OC CH indicators
AF OC CH indicators JY
KS
70 103 41 214 19 11.26
59 65 24 148 18 8.22 Module 1
Module 1
36 73 28 137 19 7.21
43 46 23 112 16 7.00 Module 2
Module 2
20 71 24 115 18 6.39
39 72 22 133 16 8.31 Module 3
Module 3
40 63 29 132 17 7.76
73 122 68 263 16 16.44 Module 4
Module 4
7 23 8 38 17 2.24
32 63 43 138 15 9.20 Module 5
Module 5
Total 173 333 130 636
Total 246 368 180 794
Note: All numbers reflect negotiated occurrences of
indicator.

25. Previous Research
• Swan (2003) found that Affective (AF) and
Open Communication (OC) increase over
time while Cohesive (C) indicators
decrease.
• Vaughan (2005) found that CH increases
over time while AF and OC decrease.
• Garrison & Arbaugh (2007) indicate that SP
should become more transparent over time.
Findings
• All 3 indicators decreased in Course JY
• AF slowly declined, and OC and CH
increased in course KS.
• Preliminary findings on SP support all three
aforementioned patterns.
JY # of stud. KS # of stud.
Module 1 19 Module 1 18
Module 2 19 Module 2 16
Module 3 18 Module 3 16
Module 4 17 Module 4 16
Module 5 17 Module 5 15
Note: All numbers reflect negotiated occurrences of
indicator.

26. Research Questions
How does:
• instructor teaching presence correlate with
student social presence?
• instructor social presence correlate with
student social presence?
• instructor presence correlate with student
presence?

27. Instructor TP – Student SP

28. Instructor SP – Student SP

29. Total Posts: Teacher - Student

30. Conclusions
• Student presence tracks instructor
presence.
• Is this a better predictor of total
interaction?
• Instructor SP is a better predictor of
student SP
• New direction for survey research?
• Ask students to assess instructor SP as
well as or instead of TP?

31. Cluster Analysis
• Segment survey respondents into
• Low, medium, high
• TP, SP, CP
• Look for patterns and associations

32. Cluster Analysis
Item Clusters
Low Medium High
The instructor communicated course topics -.90 -.05 .75
The instructor communicated course goals -.89 -.08 .74
The instructor provided clear instructions -.89 -.04 .75
The instructor communicated due dates -.84 -.06 .67
The instructor helped students learn -1.05 -.07 .85
The instructor helped students clarify their thinking -1.05 -.05 .85
The instruct or kept students engaged & participating -1.04 -.06 .85
The instructor kept students on task -1.04 -.07 .87
The instructor encouraged students to explore new ideas -1.08 -.08 .83
The instructor established a sense of course community -1.01 -.07 .86
The instructor helped focus discussion on issues that aided student -.97 -.08 .80
learning
The instructor gave feedback that helped students -1.03 -.07 .82
My instructor provided explanations or demonstrations to help me -1.06 -.06 .84
better understand the content of the course.
My instructor provided feedback to the class during the -1.04 -.06 .81
discussions or other activities to help us learn.
My instructor asked for feedback on how this course could be -.87 -.05 .76
improved.
Total Number of Student s 520 1422 1682

33. TP and SP Correlate with CP
• Evidence of an equivalence model
• Increase in SP correlates with higher CP
• But better TP compensates for lower SP

34. CP X TP X SP and Type of Instruction

35. Significant Interaction
• Student who report low or medium TP in
online courses report lower CP than like
students in hybrid courses.
• Students who report high TP in full online
courses report higher CP than student
who report high TP in hybrid courses

36. Interaction

37. Social Network Analysis
• Method to determine the location of
participants within their network of
interactions
• Measures of centrality, prestige, density
etc
• Can reveal who is contributing to
discourse and learning in significant ways

38. SNA: JY Module 1

39. SNA: Measures
Density
0.45
0.4
Density (matrix average)
0.35
0.3
0.25 KS
JY
0.2
0.15
0.1
0.05
0
M01 M02 M03 M04 M05
Modules

40. Other equivalencies
• Is SNA density roughly equivalent to
student social presence measure?
• If so, an incredible time saving
device…
• Can be computed automatically versus
• Weeks of coding in QCA

41. Density
0.45
0.4
Density (matrix average)
0.35
0.3
0.25 KS
JY
0.2
0.15
0.1
0.05
0
M01 M02 M03 M04 M05
Modules

42. Social Network Visualizations

43. Next steps
• Coding TP and SP outside of discussions
• Coding CP
• Further dialogue between survey research
and QCA