Peer Feedback in an Online Environment

Running head: CRITICAL REVIEW OF PEER FEEDBACK
Critical Review 2: Using Peer Feedback to Enhance the Quality of Student Online
Postings: An Exploratory Study
Jennifer Chang Wathall Student ID 663027
Central Michigan University
EDU 800
Instructor: Dr. Michael Dennis Deschryver

CRITICAL REVIEW OF PEER FEEDBACK TO ENHANCE QUAILITY OF POSTINGS

2
Problem (about 1 page)
1. Identify the clarity with which this article states a specific problem to be explored.
This article somewhat clearly stated the problem to be explored which was to
investigate peer feedback to enhance the quality of the postings in an online learning
environment. To achieve this, the authors investigated students’ perceptions about the value
of giving and receiving peer feedback in an online course and the quality of student online
postings. These postings were judged using Bloom’s taxonomy from pre-course to post-
course.
2. Comment on the need for this study and its educational significance as it relates
to this problem.
Many studies have looked at the impact of peer feedback in a traditional classroom
setting; however, there are relatively few studies conducted to look at the effects of peer
feedback in an online environment. Research has shown that the quality of feedback can be
improved through the use of constructive feedback providing it is prompt, consistent, and
ongoing (Ertner & Stepich, 2004). Both Lynch (2002) and Pallof and Pratt (2001) reported
that a lack of feedback in an online environment can lead to higher student dropout rate and
a sense of disconnect from the course. Furthermore, Cunningham (1992) reported that
interactions in an online environment is crucial to promote learning within the community.
Lang (as cited in Black, 2005) emphasized the importance of discourse to improve
understandings of a topic or issue. According to Black (2005), online discussions are often
at a superficial level with little critical analysis or higher order thinking. This study looked
at peer feedback as a way to promote higher order thinking by encouraging students to give
feedback to each other about the quality of their postings. The need to investigate how peer


3
feedback can enhance the quality of online posting is educationally significant for the
reasons stated above.
3. Comment on whether the problem is “researchable”? That is, can it be investigated
through the collection and analysis of data?
Both quantitative and qualitative data was collected to investigate students’
perceptions on the value of feedback and the quality of the postings. Students perceptions of
peer feedback in online courses is researchable, and surveys were conducted to collect
qualitative data. The quality of the postings was measured against a numeric scale using
Bloom’s taxonomy and quantitatively measured using t-tests.
Theoretical Perspective and Literature Review (about 3 pages)
4. Critique the author’s conceptual framework.
The authors’ conceptual framework is based on several well- researched and established
learning theories such as Vygotsky’s (1978) Zone of Proximal Development (ZPD). ZPD
describes learners’ understanding as resulting from inter-personal interactions where
students are given opportunities by the instructor to construct their own meaning of the
content through these interactions—in other words, providing a constructivist environment.
This is aligned with other perspectives that support the authors’ conceptual framework and
includes Nicol and Marfarlane’s (2006) seven functions of good feedback, which are:
· Give specific and clear goals and criteria
· Provide opportunities for self-reflection and the developing self-assessment
· Give quality comments about student learning
· Encourage peer and teacher discourse
· Be positive and supportive in your comments that motivate learners


4
·Provide clear directions to students so they know how to close the gap between
current achievement and goals
· Utilize formative assessment to inform instruction
Additionally, Cunningham (1992) discussed the importance of interaction in an online
environment to promote learning while Notar, Wilson, and Ross (1992) reported that
feedback in an online environment should be “diagnostic and prescriptive, formative and
iterative, and involving both peers and group assessment” (p. 646).
Ertmer et al., (2007) have reported that the quality of student discussions in an online
environment can be increased by “the use of constructive feedback that is prompt, consistent
and ongoing” (p. 414). This implies that online feedback may promote higher order thinking
by improving quality of postings in peer feedback. This study also used the conceptual
framework of Bloom’s taxonomy to assess the quality of online postings by the students.
Liu, Lin, Chiu, and Yuan (2001) suggested when students are asked to provide peer
feedback in an online environment they engage in higher order thinking by going beyond
cognitive processes required for a task.
The final part to the authors’ conceptual framework includes the advantages of using
peer feedback in terms of community building and student perceptions about the value of
giving and receiving peer feedback (Corgan, Hammer, Margolies, & Crossley, 2004). Pallof
and Pratt (1999) reported that giving feedback is not a naturally acquired skill. Students
often experience anxiety about giving and receiving feedback so clear guidance should be
provided.
5. How effectively does the author tie the study to relevant theory and prior research?
Are all cited references relevant to the problem under investigation?
The authors cited references and research about the following:


5
• the importance of interaction in online courses by Cunningham (1992). This
research is relevant to the problem under investigation since peer feedback can
promote more online interaction.
• the value of discourse and whether online or face to face, can lead to “increasingly
sound, well grounded, and valid understandings of a topic or issue” according to
Lang (as cited in Black, 2005, n.p.). This research is relevant to this exploratory
study as this refers to the quality of the postings as a way to reflect improved
understandings as a result of feedback.
• many online discussions according to Black (2005) lack critical analysis or higher
order thinking and mainly relate to sharing and comparing information on a
superficial level. This study used peer feedback to promote higher order thinking
where students provided feedback to each other. This related to the quality of the
postings with the goal to encourage students to learn and co-construct knowledge
and understanding (Roehler & Cantlon, 1997).
• Nicol and Marfarlane’s (2006) seven functions of good feedback are mentioned and
listed in the section above.
• According to Swartz and White (as cited in Mory, 2004), students expect feedback to
be: 1) prompt, timely and thorough 2) ongoing including summative and formative 3)
constructive, supportive and substantive 4) specific objective and individual 5)
consistent. This study ties in the need for peer feedback to support the five
expectations as an instructor alone, would be struggling to meet all of these.
• Ko and Rossem (2001) reported that a lack of feedback can result in learners feeling
disconnected from a course. The authors did not discuss the role of feedback to build


6
a sense of community in their actual investigation but this research may be indirectly
related to students’ perceptions about giving and receiving feedback.
• Ertmer and Stepich (2004) found “that the quality of student discussion responses can
be increased through the use of constructive feedback that is prompt, consistent, and
ongoing.” (as cited in Ertmer et al., n.p.). The research study supports the authors’
investigation about whether peer feedback can increase the quality of student online
postings.
6. Does the literature review conclude with a brief summary of the literature and its
implications for the problem investigated?
There is a very brief summary on page 416 stating the fact that there is limited
research on using peer feedback as an instructional strategy in an online learning
environment and that very few studies have looked at peer feedback as a way to improve the
quality of online discourse. The problem investigated, therefore, was an attempt to fill this
gap in research by investigations students’ perceptions on the value of feedback, and to see
whether peer feedback can enhance the quality of students’ online postings. There is no
brief concluding summary of the literature review.
7. Evaluate the clarity and appropriateness of the research questions or hypotheses.
Three research questions were addressed in this case study, and they were clearly
stated as:
RQ1: What is the impact of peer feedback on the quality of students’ postings in an online
environment? Can the quality of discourse/learning be maintained and/or increased through
the use of peer feedback? This research questions directly relates to the problem being
investigated which is whether the quality of student online postings can be improved with
the use of peer feedback.


7
RQ2: What are students’ perceptions of the value of receiving peer feedback? How do
these perceptions compare to the perceived value of receiving instructor feedback? I am not
sure how relevant this is to investigating the role of peer feedback in enhancing the quality
of online discussions. Students’ perceptions are important, however, I cannot see the
connection to how this research question directly relates to main issue discussed which was
to investigate whether peer feedback can improve the quality of online postings.
RQ3: What are students’ perceptions of the value of giving peer feedback? Again, I am not
sure how this question is directly related to investigating improving the quality of online
postings and promoting higher order thinking in an online environment. There are only a
couple of research articles cited in this article that relate to student perceptions on feedback,
which are described by Topping (1998) and Arbaugh (2000).
Research Design and Analysis (about 3 pages)
8. Critique the appropriateness and adequacy of the study’s design in relation to the
research questions or hypotheses.
This study represented a case study framework with the use of qualitative data,
however, the study also collected quantitative data and used an experimental research design
(Randolph, 2007). According to Yin (2003), “a case study is an empirical inquiry that
investigates a contemporary phenomenon within its real-life context, especially when the
boundaries between the phenomenon and context are not clearly evident” (p. 13). Several
sources of evidence are normally collected through direct observation, interviews for
example, and then analyzed through pattern making and explanation building. In this study
students were given feedback by the instructor for the first half of the course so students
could see examples of quality feedback. In the last half of the course, students were required


8
to give peer feedback to two discussion posts and answer one discussion questions
themselves.
Three research questions were addressed:
1. What is the impact of peer feedback on the quality of students’ postings in an online
environment? Can the quality of discourse/learning be maintained and/or increased
through the use of peer feedback? The case study framework is well suited to address
this question. The experimental design for the collection of quantitative data used a
numerical score based on Bloom’s taxonomy, as well as descriptive comments that
supported the assigned score.
2. What are students’ perceptions of the value of receiving peer feedback? How do these
perceptions compare to the perceived value of receiving instructor feedback? Data
collected here was based on participant interviews and responses to entry and exit
survey questionnaires which included 13 Likert- style items and 5 open-ended questions.
3. What are students’ perceptions of the value of giving peer feedback? Again, data
collected here was based on participant interviews, and responses to entry and exit
survey questionnaires which had a combination of both quantitative and qualitative data.
9. Critique the adequacy of the study’s sampling methods (e.g., choice of participants)
and their implications for generalizability.
The sample of 15 graduate students that was selected did not represent a random
sample and is considered too small to be representative of the population. Most
recommendations for qualitative data collection that will undergo statistical analysis is to
have at least 30 data points (Morse, 1994). This small sample cannot be taken to be
representative of the entire population in terms of generalizability and also could be
described as a “convenience sample” as participants were all graduate students enrolled in


9
the same online course (Given, 2008). This study used a t test, which has a specific
calculation for the recommended sample size that involves looking at standardized effect
sizes (Hulley et al., 2013). This computation was not used to calculate the recommend
sample size for this particular study and may lead to low statistical power.
10. Critique the adequacy of the study’s procedures and materials (e.g., interventions,
interview protocols, data collection procedures).
To collect quantitative data about the quality of student online postings, two
researchers rated the students’ postings separately and then verbally discussed their scores.
The raters looked at 59 different postings and there was an 86% agreement between the two
raters which reflects a high level of agreement. The authors used a paired sample t test to
compare the average scores obtained prior to peer feedback (weeks 3-5) and the scores after
peer feedback (weeks 7-13).
T tests were also employed to compare students’ rating of the value of peer and
instructor feedback from the pre and post surveys. Participants perceptions about the value
of peer feedback used data from open ended survey questions and interviews. Participants
interviews were conducted to obtain qualitative data about students’ perceptions about the
peer feedback process.
11. Critique the appropriateness and quality (e.g., reliability, validity) of the measures
used.
Reliability of the measures used refers to the consistency of the measurements and if the
experiment was to be repeated whether the experiment would yield the same results. There
are some reliability issues in terms of the small number of participants. This sampling
problem can cause a random error in the measurements and data collected and if this
experiment were to be repeated, it could yield completely different results. The most


10
common measure of reliability is using the reliability coefficient and this study did not
include this calculation to check for reliability (Rosnow & Rosenthal, 1991). Typical
methods in social research include: test-retest, alternative forms, split halves, inter-rater
reliability and internal consistency. There is no mention in the study of measuring the
internal consistency using Cronbach’s alpha coefficient (Cronbach, 1951). The first two
limitations of the small sample size of 15 graduate students used and the short timing of the
course all call into question the reliability of the results.
Validity is concerned with how well does a test measure what the researchers have
set out to measure and can relate to the credibility of qualitative research. There are four
types of validity: statistical conclusion validity, internal validity, construct validity and
external validity (Drost, 2011). The limited scale of Bloom’s taxonomy used to measure the
quality of students’ posting, may affect the validity of the results. Did the limited scale
actually measure the quality of the feedback postings? The authors stated they had a
relatively high degree of face validity (within construct validity) but then contradict
themselves in the conclusion by stating this was one of the limitations of the study.
This study is lacking construct validity, which refers to how well an idea had been
transformed or translated into a “functioning and operating reality” (Drost, 2011). Did the
limited scale used actually measure the quality of the feedback postings? How do we know
Bloom’s taxonomy for scoring the quality accurately measured higher order thinking and
the quality postings?
C. Interpretation and Implications of Results (about 3 pages)
13. Critique the author’s discussion of the methodological and/or conceptual
limitations of the results.
The authors reported the results and limitations as following:


11
• At the beginning of the course, feedback in the online environment
was found to be slightly more important than traditional settings and
students reported the feedback should be timely and high quality
• Students were not so concerned about the quantity of feedback.
• By the end of the course, students’ perceptions about the importance of
giving and receiving feedback significantly increased.
• Students reported having concerns about giving feedback in terms of
consistency, fairness and not being qualified to give feedback.
• A paired t test revealed no significance difference in terms of the
quality of students’ postings either with instructor or peer feedback.
Even though there was no increase in quality, authors pointed out
there was no decrease in the quality of student postings, suggesting
that peer feedback can maintain the quality of postings once a level
has been reached. This could be a benefit to the instructor in terms of
saving time and reducing work load.
• Another limitation mentioned by the authors is the actual discussion
questions used in the course. Many of the questions were devised by
student leaders and did not elicit a lot of higher order thinking, so
ratings for responses could have scored low on Bloom’s taxonomy
due to this reason. In summary, discussion questions did not
encourage responses that connected to the higher levels of thinking
which involve analysis, synthesis or evaluation.
• Many students also started off with relatively high scores for the quality
of their postings, which leaves very little room for improvement.


12
• Data collected from interviews suggest that students did use peer
feedback information to improve the quality of their postings.
• Instructor feedback was perceived to be more valuable than peer
feedback and this has been reported by several researchers such as Ko
and Rossen, (2001) and Topping (1998).
The three limitations mentioned include:
• the small sample size of 15 graduate students used
• the short timing of the course
• the limited scale used to assess the quality of the student postings
The limited scale used Bloom’s taxonomy to assess the quality of postings resulted in
several students reporting the taxonomy was difficult to apply, too vague and not
appropriate to use at a graduate level.
14. How consistent and comprehensive are the author’s conclusions with the reported
results?
The authors’ conclusions are fairly consistent with the reported results. The
conclusions drawn were that online discussions play an important role in maintaining the
quality of the postings and can reduce instructors’ work load without any negative impacts
on student learning. Two research questions looked at students’ perceptions about the value
of giving and receiving feedback and the authors reported that students valued both giving
and receiving feedback as equally important and student perceptions about the importance
of feedback significantly increased over the duration of the course. Results showed that
students do value the importance of giving and receiving feedback despite the fact they
value instructor feedback over peer feedback. The authors suggested that instructors need to
validate peer feedback responses so students can see the benefits more.


13
The initial aim of this study was to investigate whether peer feedback could
improve the quality of student online postings. The findings showed there was no increase
and no decrease in the quality of postings with the introduction of peer feedback. Since the
quality of the postings were maintained during the course, the authors suggested that peer
feedback could be used to manage instructors’ workload without negatively impacting
student learning. This finding relates to the study by Swartz and White (as cited in Mory,
2004) about the use of peer feedback to help manage feedback expectations which are that
feedback should be: 1) prompt, timely and thorough; 2) ongoing including summative and
formative; 3) constructive, supportive and substantive; 4) specific objective and individual;
and 5) consistent.
Based on the results of this study the authors recommend seven areas to consider
when designing an online course and they are:
• Help learners understand why and how the feedback process works
• As an instructor model and give examples of effective feedback
• Provide structure and guidelines to leaners about giving and receiving
feedback
• As a facilitator monitor the peer feedback process by providing feedback on
the peer feedback
• When possible ensure the feedback is anonymous
• Use multiple peer ratings to have an overview of the value of learners’
responses
• Make the peer feedback process easy to follow and straight forward
15. How well did the author relate the results to the study’s theoretical base?


14
The authors’ results supported the theoretical base that peer feedback can be a
beneficial instructional strategy in an online environment and allows students to critically
analyze their postings and self-reflect on what a high-quality posting looks like (Henderson,
Rada & Chen, 1997; Topping, 1998). Topping (1998) also found that even though peer
feedback was not as high quality compared with from an instructor, the immediacy, greater
frequency and volume compensates for this. In this study, Ertmer et al. (2007) found that
students are not concerned with the quantity of feedback, but more the quality of the
feedback, which is contradictory to Topping’s (1998) findings. The immediacy in this case
study was not met, as feedback sometimes took as long as two weeks, which counters one of
the positive effects of peer feedback. Despite its limitations, utilizing Bloom’s taxonomy
encouraged students to reflect on higher order thinking such as providing justifications and
providing valuable critique.
The authors reported one of the greatest potential benefits of peer feedback is how
students are encouraged to construct their own meaning and understanding of peer feedback
which relates to the theoretical base of Vygotsky’s (1978) constructivism and Zone of
Proximal Distance.
16. In your view, what is the significance of the study, and what are its primary
implications for theory, future research, and practice?
The significance of this study was the idea that the power of using peer feedback in
an online environment to maintain the quality of students’ postings, reduce the work load of
the instructor and to help learners self-reflect, and engage in higher order thinking. The
authors also found that students’ perceptions of feedback are that it should be timely and be
of high quality rather than high quantity. The seven recommendations about feedback from
this study include practical strategies to use feedback effectively in an online environment.


15
Future research could investigate the benefits of peer feedback as a community
building strategy and one that promotes collegiality and trust and ultimately increasing
motivation for individuals in an online environment thus improving learning and retention
rates (van Ginkel, Gulikers, Biemans & Mulder, 2017; Espitia & Cruz Corzo, 2013). Future
research could also investigate what models of effective feedback could be used to teach
learners how to give and receive feedback and what factors do actually improve the quality
of student online posting with the introduction of peer feedback. Different metrics could be
employed to judge the quality of postings to measure the critical thinking and analysis and
also better online discussion prompts could be devised to provide learners with the
opportunity to demonstrate their higher order thinking.


16
References
Black, A. (2005). The use of asynchronous discussion: Creating a text of talk. Contemporary
Issues in Technology and Teacher Education, 5(1). Retrieved October 3, 2005 from http://
www.citejournal.org/vol5/iss1/languagearts/article1.cfm
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3),
297-334.
Cunningham, R. D. (1992). Beyond educational psychology: Steps toward an educational
semiotic. Educational Psychology Review, 4, 165–194.
Drost, Ellen. (2011). Validity and reliability in social science research. Education Research and
Perspectives, 38, 105-124.
Ertmer, P. A., & Stepich, D. A. (2004, July). Examining the relationship between higher-order
learning and students’ perceived sense of community in an online learning environment.
Paper presented at the proceedings of the 10th Australian World Wide Web conference,
Gold Coast, Australia.
Ertmer, P. A., Richardson, J. C., Belland, B., Camin, D., Connolly, P., Coulthard, G., . . . Mong, C.
(2007). Using peer feedback to enhance the quality of student online postings: An
exploratory study. Journal of Computer-Mediated Communication, 12(2), 412-433.
doi:10.1111/j.1083-6101.2007.00331.x
Espitia, M., & Cruz Corzo, C. (2013). Peer-feedback and online interaction: A case study. Ikala,
Revista De Lenguaje Y Cultura, 18(2), 131-151.
Given, L. (2008). "Convenience Sample". The SAGE Encyclopedia of Qualitative Research
Methods. SAGE Publications. doi:10.4135/9781412963909.n68.
Henderson, T., Rada, R., & Chen, C. (1997). Quality management of student-student
evaluations. Journal of Educational Computing Research, 17, 199–215.
Hulley, S. B., Cummings, S. R., Browner, W. S., Grady, D. G., & Newman, T. B. (2013).


17
Designing clinical research. Philadelphia: Wolters Kluwer Health / Lippincott Williams
& Wilkins.
Ko, S., & Rossen, S. (2001). Teaching Online: A Practical Guide. Boston, MA:
Houghton-Mifflin.
Lynch, M. M. (2002). The Online Educator: A Guide to Creating the Virtual Classroom.
New York: RoutledgeFalmer.
Morse, J. M. (1994). Designing funded qualitative research. In Denizin, N. K. & Lincoln, Y.
S., Handbook of qualitative research (2nd Ed). Thousand Oaks, CA: Sage.
Mory, E. H. (2004). Feedback research revisited. In D. H. Jonassen (Ed.), Handbook of
Research on Educational Communications and Technology (pp. 745–783). Mahwah,
NJ: Lawrence Erlbaum.
Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning:
A model and seven principles of good feedback practice. Studies in Higher Education,
31(2), 199–218.
Palloff, R. M., & Pratt, K. (1999). Building Learning Communities in Cyberspace. San
Francisco, CA: Jossey-Bass.
Palloff R. M., & Pratt, K. (2001). Lessons from the Cyberspace Classroom: The Realities of Online
Teaching. San Francisco, CA: Jossey-Bass.
Randolph, Justus J. (2007). Multidisciplinary Methods in Educational Technology Research and
Development, HAMK Press, HTML (chapters 1 & 2).
Roehler, L. R., & Cantlon, D. J. (1997). Scaffolding: A powerful tool in social constructivist
classrooms. In K. Hogan & M. Pressley (Eds.), Scaffolding Student Learning: Instructional
Approaches and Issues (pp. 6–42). Cambridge, MA: Brookline.


18
Rosenthal, R. and Rosnow, R. L. (1991). Essentials of Behavioral Research: Methods and Data
Analysis. Second Edition.
Topping, K. (1998). Peer assessment between students in colleges and universities. Review of
Educational Research, 68(3), 249–276.
van Ginkel, S., Gulikers, J., Biemans, H., & Mulder, M. (2017). Fostering oral presentation
performance: Does the quality of feedback differ when provided by the teacher, peers or
peers guided by tutor? Assessment & Evaluation in Higher Education, 42(6), 953-966.
Vygotsky, L. S. (1978). Mind in Society: The Development of Higher Psychological Processes.
Cambridge, MA: Harvard University Press.
Yin, R. K. (2003). Case Study Research: Design and Methods (3rd ed.). Thousand Oaks, CA: Sage.

Peer Feedback in an Online Environment

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (16)

Semelhante a Peer Feedback in an Online Environment

Semelhante a Peer Feedback in an Online Environment (20)

Mais de Dr. Jennifer Chang Wathall

Mais de Dr. Jennifer Chang Wathall (20)

Último

Último (20)

Peer Feedback in an Online Environment