O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
EksoBionicsflickr.com
Using technology
to provide
feedback on
learning strategies
Abelardo Pardo (@abelardopardo)
Faculty o...
Abelardo Pardo Using technology to provide feedback on learning strategies 2
MichaelCoghlanflickr.com
About

Feedback Feedb...
Abelardo Pardo Using technology to provide feedback on learning strategies 3
MichaelCoghlanflickr.com
About

Feedback
Abelardo Pardo Using technology to provide feedback on learning strategies
"information provided by an agent (teacher, pee...
Abelardo Pardo Using technology to provide feedback on learning strategies
Innovations designed to strengthen the frequent...
Abelardo Pardo Using technology to provide feedback on learning strategies
If you could choose one…
• More than 500 meta-a...
Abelardo Pardo Using technology to provide feedback on learning strategies 7
Feedback is inherent in and a
prime determine...
Abelardo Pardo Using technology to provide feedback on learning strategies
Feedback models
1. Task Level (understanding,
p...
Abelardo Pardo Using technology to provide feedback on learning strategies
What is the recipe?
1. encourage contact betwee...
Abelardo Pardo Using technology to provide feedback on learning strategies 10
Hounsell, D. (2007). Toward more sustainable...
Abelardo Pardo Using technology to provide feedback on learning strategies 11
Hounsell, D. (2008). The Trouble with Feedba...
Abelardo Pardo Using technology to provide feedback on learning strategies
NicolasNovaflickr.com
Feedback is inherently pro...
Abelardo Pardo Using technology to provide feedback on learning strategies
1) Construct an understanding
2) Use it to regu...
Abelardo Pardo Using technology to provide feedback on learning strategies 14
MichaelCoghlanflickr.com
About

Feedback Feed...
Abelardo Pardo Using technology to provide feedback on learning strategies 15
JoelSageflickr.com
“sustainability of feedbac...
Abelardo Pardo Using technology to provide feedback on learning strategies 16
AllenLaiflickr.com
Hounsell, D. (2007). Towar...
Abelardo Pardo Using technology to provide feedback on learning strategies
Feedback is delicate. It must be positive. High...
Abelardo Pardo Using technology to provide feedback on learning strategies
The feedback question gets systematically
lower...
Abelardo Pardo Using technology to provide feedback on learning strategies
• Ambiguity or opacity (What do you
mean?)
• Ne...
Abelardo Pardo Using technology to provide feedback on learning strategies 20
PostmodernSleazeflickr.com
Boud, D., & Molloy...
Abelardo Pardo Using technology to provide feedback on learning strategies 21
Active Learning Works
Abelardo Pardo Using technology to provide feedback on learning strategies 22
2 Hours
2 Hours
3 Hours
Pardo, A. & Mirriahi...
Abelardo Pardo Using technology to provide feedback on learning strategies 23
Students are less likely to
engage in pre-cl...
Abelardo Pardo Using technology to provide feedback on learning strategies 24
MichaelCoghlanflickr.com
About

Feedback Feed...
Abelardo Pardo Using technology to provide feedback on learning strategies
Instructive or process feedback is not common i...
Abelardo Pardo Using technology to provide feedback on learning strategies 26
HungarianSnowflickr.com
Intelligence Amplifica...
Abelardo Pardo Using technology to provide feedback on learning strategies 27
Baker, R. S. (2016). Stupid Tutoring Systems...
Abelardo Pardo Using technology to provide feedback on learning strategies 28
BrandonMartin-Andersonflickr.com
• Can data g...
Abelardo Pardo Using technology to provide feedback on learning strategies 29
JoshRepennekerFlickr.com
Can technology and
...
Abelardo Pardo Using technology to provide feedback on learning strategies
Simplistic View Emerging
“We are really shiftin...
Abelardo Pardo Using technology to provide feedback on learning strategies 31
MichaelCoghlanflickr.com
About

Feedback Feed...
Abelardo Pardo Using technology to provide feedback on learning strategies 32
• First year engineering
course
• Computer S...
Abelardo Pardo Using technology to provide feedback on learning strategies 33
2 Hours
2 Hours
3 Hours
Pardo, A. & Mirriahi...
Abelardo Pardo Using technology to provide feedback on learning strategies 34
Abelardo Pardo Using technology to provide feedback on learning strategies 35
1. realtime feedback
2. convey engagement
3....
Abelardo Pardo Using technology to provide feedback on learning strategies 36
Tracking
Abelardo Pardo Using technology to provide feedback on learning strategies 37
Technology
Technology
Week N
Abelardo Pardo Using technology to provide feedback on learning strategies 38
Weekly real time feedback
Abelardo Pardo Using technology to provide feedback on learning strategies 39
No statistically significant difference in th...
Abelardo Pardo Using technology to provide feedback on learning strategies 40
1. encourage contact
between student and
fac...
Abelardo Pardo Using technology to provide feedback on learning strategies 41
Week N
Technology
Human
Abelardo Pardo Using technology to provide feedback on learning strategies 42
Pardo, A., Mirriahi, N., Martínez-Maldonado,...
Abelardo Pardo Using technology to provide feedback on learning strategies 43
You should take a more
careful look at how s...
Abelardo Pardo Using technology to provide feedback on learning strategies 44
Algorithm
Abelardo Pardo Using technology to provide feedback on learning strategies 45
Automatic
Email
Abelardo Pardo Using technology to provide feedback on learning strategies 46
1. Data collected weekly
2. Email sent at en...
Abelardo Pardo Using technology to provide feedback on learning strategies 47
Significant difference in the rating of feedb...
Abelardo Pardo Using technology to provide feedback on learning strategies 48
Significant difference in the midterm score (...
Abelardo Pardo Using technology to provide feedback on learning strategies 49
MatthewandHeaderflickr.com
Feedback: high imp...
EksoBionicsflickr.com
Using technology
to provide
feedback on
learning strategies
Abelardo Pardo (@abelardopardo)
Faculty o...
Próximos SlideShares
Carregando em…5
×

Using technology to provide feedback on learning strategies

3.548 visualizações

Publicada em

Feedback is problematic, and in higher education the situation is degrading quickly due to pressures to cater to massive cohorts while maintaining effectiveness. Technology can help achieve scalable feedback, and we show a few examples illustrating the process.

Publicada em: Educação
  • Seja o primeiro a comentar

Using technology to provide feedback on learning strategies

  1. 1. EksoBionicsflickr.com Using technology to provide feedback on learning strategies Abelardo Pardo (@abelardopardo) Faculty of Engineering and IT slideshare.net/abelardo_pardo Digital Learning Week
 University of South Australia
 16 March 2016
  2. 2. Abelardo Pardo Using technology to provide feedback on learning strategies 2 MichaelCoghlanflickr.com About
 Feedback Feedback
 Higher Ed Tech to scale
 feedback Examples
  3. 3. Abelardo Pardo Using technology to provide feedback on learning strategies 3 MichaelCoghlanflickr.com About
 Feedback
  4. 4. Abelardo Pardo Using technology to provide feedback on learning strategies "information provided by an agent (teacher, peer, book, parent, self, experience) regarding aspects of one’s performance or understanding” 4 Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81-112. doi: 10.3102/003465430298487 FarukAteşflickr.com
  5. 5. Abelardo Pardo Using technology to provide feedback on learning strategies Innovations designed to strengthen the frequent feedback that students receive about their learning yield substantial learning gains 5 Black, P., & Wiliam, D. (1998). Assessment and Classroom Learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7-74. doi:10.1080/0969595980050102 GarethChristopherflickr.com
  6. 6. Abelardo Pardo Using technology to provide feedback on learning strategies If you could choose one… • More than 500 meta-analyses of student achievements • 100 factors with potential influence • Feedback in top five • (74 meta-analyses) Most effective form: video, audio, computer-assisted instructional feedback, and/or related goals 6 Hattie, J. A. (1999). Influences on student learning. Inaugural professorial address, University of Auckland, New Zealand
  7. 7. Abelardo Pardo Using technology to provide feedback on learning strategies 7 Feedback is inherent in and a prime determiner of processes that constitute self regulated learning Galflickr.com Butler, D. L., & Winne, P. H. (1995). Feedback and Self-Regulated Learning: A Theoretical Synthesis. Review of Educational Research, 65(3), 245-281
  8. 8. Abelardo Pardo Using technology to provide feedback on learning strategies Feedback models 1. Task Level (understanding, performance) 2. Process Level (what to do to understand, perform) 3. Self-regulation level (detecting and directing effort) 4. Self level (personal evaluation and affect) 8 Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81-112. doi: 10.3102/003465430298487 Orkomedixflickr.com
  9. 9. Abelardo Pardo Using technology to provide feedback on learning strategies What is the recipe? 1. encourage contact between student and faculty 2. develops reciprocity and cooperation among students 3. uses active learning techniques 4. gives prompt feedback 5. emphasizes time on task 6. communicates high expectations 7. respects diverse talents and ways of learning 9 Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. American Association for Higher Education Bulletin, 39, 3-7. 
 Chickering, A. W., & Ehrmann, S. C. (1996). Implementing the seven principles: Technology as lever. American Association for Higher Education Bulletin, 49, 3-6.
  10. 10. Abelardo Pardo Using technology to provide feedback on learning strategies 10 Hounsell, D. (2007). Toward more sustainable feedback to students. In D. Boud & N. Falchikov (Eds.), Rethinking Assessment in Higher Education: Learning for the Longer Term. London and New York: Routledge. Perceived as an administrative chore 
 instead of a pedagogical necessity MarcinWicharyflickr.com
  11. 11. Abelardo Pardo Using technology to provide feedback on learning strategies 11 Hounsell, D. (2008). The Trouble with Feedback. New Challenges, Emerging Strategies. Interchange(2). Feedback has a Cinderella status Bitslammerflickr.com
  12. 12. Abelardo Pardo Using technology to provide feedback on learning strategies NicolasNovaflickr.com Feedback is inherently problematic 12 Higgins, R., Hartley, P., & Skelton, A. (2010). Getting the Message Across: The problem of communicating assessment feedback. Teaching in Higher Education, 6(2), 269-274. doi:10.1080/13562510120045230 Feedback as communication
  13. 13. Abelardo Pardo Using technology to provide feedback on learning strategies 1) Construct an understanding 2) Use it to regulate 13 Nicol, D. J., & Macfarlane‐Dick, D. (2006). Formative assessment and self‐regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218. doi:10.1080/03075070600572090 JamesButlerflickr.com
  14. 14. Abelardo Pardo Using technology to provide feedback on learning strategies 14 MichaelCoghlanflickr.com About
 Feedback Feedback
 Higher Ed
  15. 15. Abelardo Pardo Using technology to provide feedback on learning strategies 15 JoelSageflickr.com “sustainability of feedback is under threat… pervasive student concerns about the provision of feedback in an era of mass education" Hounsell, D. (2007). Toward more sustainable feedback to students. In D. Boud & N. Falchikov (Eds.), Rethinking Assessment in Higher Education: Learning for the Longer Term. London and New York: Routledge.
  16. 16. Abelardo Pardo Using technology to provide feedback on learning strategies 16 AllenLaiflickr.com Hounsell, D. (2007). Toward more sustainable feedback to students. In D. Boud & N. Falchikov (Eds.), Rethinking Assessment in Higher Education: Learning for the Longer Term. London and New York: Routledge. Downward spiral: Student disenchantment mounts when feedback is uninformative and 
 as a consequence, less feedback is provided Solution: provide high-value feedback, rethink the role of student, enhance relation between guidance and feedback
  17. 17. Abelardo Pardo Using technology to provide feedback on learning strategies Feedback is delicate. It must be positive. High fragility of confidence felt by many students 17 Ferguson, P. (2011). Student perceptions of quality feedback in teacher education. Assessment & Evaluation in Higher Education, 36(1), 51-62. doi:10.1080/02602930903197883 Doranflickr.com
  18. 18. Abelardo Pardo Using technology to provide feedback on learning strategies The feedback question gets systematically lower values in student surveys 18 Krause, K.-L., Hartley, R., James, R., & McInnis, C. (2005). The First Year Experience in Australian Universities: Findings from a decade of National Studies. University of Melbourne: Centre for the Study of Higher Education. Eleafflickr.com
  19. 19. Abelardo Pardo Using technology to provide feedback on learning strategies • Ambiguity or opacity (What do you mean?) • Negativity 19 Huxham, M. (2007). Fast and effective feedback: are model answers the answer? Assessment & Evaluation in Higher Education, 32(6), 601-611. doi:10.1080/02602930601116946 • Lateness • Uncertainty about criteria and contexts Loonyhikerflickr.com
  20. 20. Abelardo Pardo Using technology to provide feedback on learning strategies 20 PostmodernSleazeflickr.com Boud, D., & Molloy, E. (2013). Rethinking models of feedback for learning: the challenge of design. Assessment & Evaluation in Higher Education, 38(6), 698-712. doi:10.1080/02602938.2012.691462 Conventional feedback: Mark 1 Impractical due to resources Reserved for important concepts only Could be extended by technology if
 careful selection is applied Mark 2 From mechanistic to responsive Curriculum-wide view Foster self-regulation, feedback
 seeking skills, calibrated 
 judgement, etc.
  21. 21. Abelardo Pardo Using technology to provide feedback on learning strategies 21 Active Learning Works
  22. 22. Abelardo Pardo Using technology to provide feedback on learning strategies 22 2 Hours 2 Hours 3 Hours Pardo, A. & Mirriahi N. (in press). Design, Deployment and Evaluation of a Flipped Learning First Year Engineering Course. In C. Reidsema, Kavanagh, R. Hadgraft & N. Smith (Eds.), Flipping the Classroom: Practice and Practices. Singapore: Springer.
  23. 23. Abelardo Pardo Using technology to provide feedback on learning strategies 23 Students are less likely to engage in pre-class activities if they are not interactive, do not provide formative feedback, and not coherently linked with the face-to-face activities O'Flaherty, J., & Phillips, C. (2015). The use of flipped classrooms in higher education: A scoping review. The Internet and Higher Education, 25, 85-95. doi:10.1016/j.iheduc.2015.02.002 DanKlimkeflickr.com
  24. 24. Abelardo Pardo Using technology to provide feedback on learning strategies 24 MichaelCoghlanflickr.com About
 Feedback Feedback
 Higher Ed Tech to scale
 feedback
  25. 25. Abelardo Pardo Using technology to provide feedback on learning strategies Instructive or process feedback is not common in learning analytics 25 Gasevic, D., Dawson, S., & Siemens, G. (2015). Let's not forget: Learning analytics are about learning. TechTrends, 59(1), 64-75. ChefCookeflickr.com
  26. 26. Abelardo Pardo Using technology to provide feedback on learning strategies 26 HungarianSnowflickr.com Intelligence Amplification Hierarchies of regulators to achieve amplification We measure intelligence as the capacity to make the appropriate selection Ashby, W. R. (1957). An Introduction to Cybernetics. London: Chapman & Hall Ltd.
  27. 27. Abelardo Pardo Using technology to provide feedback on learning strategies 27 Baker, R. S. (2016). Stupid Tutoring Systems, Intelligent Humans. International Journal of Artificial Intelligence in Education. doi: 10.1007/s40593-016-0105-0 Rienties, B., Boroowa, A., Cross, S., Kubiak, C., Mayles, K., & Murphy, S. (2016). Analytics4Action Evaluation Framework: A Review of Evidence-Based Learning Analytics Interventions at the Open University UK. Journal of Interactive Media in Education, 2016(1). doi:10.5334/jime.394 Change in focus, from artificial intelligence to
 intelligence amplification Tutors are informed about students struggling but it is up to them to decide if/when to act
  28. 28. Abelardo Pardo Using technology to provide feedback on learning strategies 28 BrandonMartin-Andersonflickr.com • Can data guide the provision of effective feedback • Can this provision be done at scale?
  29. 29. Abelardo Pardo Using technology to provide feedback on learning strategies 29 JoshRepennekerFlickr.com Can technology and data be used to change the economic model of feedback, achieve scale and maintain effectiveness?
  30. 30. Abelardo Pardo Using technology to provide feedback on learning strategies Simplistic View Emerging “We are really shifting the role of an educator to someone who is more of a data- enabled detective” Max Ventilla 
 AltSchool founder
 (emphasis added) 30 Mead, Rebecca (2016). Learn Different. Silicon Valley disrupts education. The New Yorker, Annals of Technology, March 7 Issue
  31. 31. Abelardo Pardo Using technology to provide feedback on learning strategies 31 MichaelCoghlanflickr.com About
 Feedback Feedback
 Higher Ed Tech to scale
 feedback Examples
  32. 32. Abelardo Pardo Using technology to provide feedback on learning strategies 32 • First year engineering course • Computer Systems • Flipped Blended Learning • ~300 students • Preparation activities with formative feedback and
 summative assessment • 42% satisfied with feedback (meh)
  33. 33. Abelardo Pardo Using technology to provide feedback on learning strategies 33 2 Hours 2 Hours 3 Hours Pardo, A. & Mirriahi N. (in press). Design, Deployment and Evaluation of a Flipped Learning First Year Engineering Course. In C. Reidsema, Kavanagh, R. Hadgraft & N. Smith (Eds.), Flipping the Classroom: Practice and Practices. Singapore: Springer. 2014 Edition
  34. 34. Abelardo Pardo Using technology to provide feedback on learning strategies 34
  35. 35. Abelardo Pardo Using technology to provide feedback on learning strategies 35 1. realtime feedback 2. convey engagement 3. compare with rest of cohort 4. reset weekly 5. one click away from notes Approach 1 (2014) SeanDreilingerflickr.com
  36. 36. Abelardo Pardo Using technology to provide feedback on learning strategies 36 Tracking
  37. 37. Abelardo Pardo Using technology to provide feedback on learning strategies 37 Technology Technology Week N
  38. 38. Abelardo Pardo Using technology to provide feedback on learning strategies 38 Weekly real time feedback
  39. 39. Abelardo Pardo Using technology to provide feedback on learning strategies 39 No statistically significant difference in the rating of feedback (2013 edition, M=3.25, SD=0.97; 2014 edition, M=3.35, SD=1.03); t(389.78) = -0.97, p <0.17
  40. 40. Abelardo Pardo Using technology to provide feedback on learning strategies 40 1. encourage contact between student and faculty (forum) 2. uses active learning techniques 3. gives prompt feedback 4. emphasizes time on task 5. kind, specific, helpful Approach 2 (2015)Ben Manson flickr.com BenMansonflickr.com
  41. 41. Abelardo Pardo Using technology to provide feedback on learning strategies 41 Week N Technology Human
  42. 42. Abelardo Pardo Using technology to provide feedback on learning strategies 42 Pardo, A., Mirriahi, N., Martínez-Maldonado, R., Jovanović, J., Dawson, S., & Gasevic, D. (2016). Generating Actionable Predictive Models of Academic Performance. Paper presented at the International Conference on Learning Analytics and Knowledge, Edinburgh. 1. Group students 2. Each group has an estimated midterm score 3. Groups are determined by conditions in engagement measures 4. Can I act on this information? Student Clustering based on Engagement
  43. 43. Abelardo Pardo Using technology to provide feedback on learning strategies 43 You should take a more careful look at how symbols are encoded in the video. Would you be able to encode/ decode UAL symbols without looking at the video? Good initial work. However, did you understand the trick to handle encoding with a variable number of bits? Would you be able to provide an example? Good work. Would you be able to come up with your own machine language and your encoding scheme? Remember that it has to be unambiguous. Thorough work with the task about machine language encoding. Give it a quick review before the midterm. Q1 Q2 Q3 Q4 Instructor
  44. 44. Abelardo Pardo Using technology to provide feedback on learning strategies 44 Algorithm
  45. 45. Abelardo Pardo Using technology to provide feedback on learning strategies 45 Automatic Email
  46. 46. Abelardo Pardo Using technology to provide feedback on learning strategies 46 1. Data collected weekly 2. Email sent at end of week 3. 4 weeks before the midterm nateOneflickr.com
  47. 47. Abelardo Pardo Using technology to provide feedback on learning strategies 47 Significant difference in the rating of feedback (2014 edition, M=3.35, SD=1.03; 2015 edition, M=3.82, SD=0.90); t(389.78) = -4.88, p <10-6
 Effect size (Cohen’s d) = 0.49. Medium positive effect
  48. 48. Abelardo Pardo Using technology to provide feedback on learning strategies 48 Significant difference in the midterm score (2014 edition, M=12.80, SD=4.79; 2015 edition, M=13.83, SD=4.89); t(684.5) = -2.86, p < 0.002
 Effect size (Cohen’s d) = 0.21. Small positive effect
  49. 49. Abelardo Pardo Using technology to provide feedback on learning strategies 49 MatthewandHeaderflickr.com Feedback: high impact/risk Higher Ed is in dire need for feedback revamp Hybrid methods: human intelligence amplification Data supported approaches allow effective scaling
  50. 50. EksoBionicsflickr.com Using technology to provide feedback on learning strategies Abelardo Pardo (@abelardopardo) Faculty of Engineering and IT slideshare.net/abelardo_pardo Digital Learning Week
 University of South Australia
 16 March 2016

×