Assessment Issues & Strategies

1. Assessment Issues &
Strategies
David Carless
University of Hong Kong
Presentation at Hong Kong Shue
Yan University, March 26, 2018
The University of Hong Kong

2. Overview
1. Learning-oriented assessment
2. Good assessment designs
3. Barriers & challenges
4. Conclusions & implications
The University of Hong Kong

3. What comes to mind when you
think of “assessment”?
The University of Hong Kong

4. Competing assessment functions
The University of Hong Kong

5. Learning-oriented assessment
 To alleviate
competing priorities
The University of Hong Kong
 circumnavigate
formative/summative divides
and

6. A focus on student learning
 By prioritizing student learning as a key aim of
all assessment
The University of Hong Kong

7. The University of Hong Kong
Productive assessment
task design
Development of student self-
evaluative capacities
Student engagement
with feedback
Learning-oriented assessment framework
(Carless, 2015)

9. The University of Hong Kong
HISTORY
EXAMPLE

10. History Assessment
Fieldwork report (30%): Museum visit
Individual project (40%): draft 10%, final 30%
Participation (30%):
tutorial participation 15%
short weekly written responses 15%
(cf. Carless & Zhou, 2015)
The University of Hong Kong

11. Museum report
1000 words or podcast
Issues: key messages; use of space;
coverage and omissions
The University of Hong Kong

12. Sample Project
Walking tour of sites of Hong Kong identity
The University of Hong Kong

13. Project stages
• Topic brief
• Draft (10%)
• Final version (30%)
The University of Hong Kong

14. Use of Social Media
Students could upload drafts of work in
progress & receive peer feedback
The University of Hong Kong

15. The University of Hong Kong
LAW EXAMPLE

16. Tort Law
• Core 1st
year course: 180+ students
The University of Hong Kong

17. Tort Law Assessment
Reflective Media Diary (20%)
1st
sem test (20%)
Final Exam (60%)
[or 40% + 20% research essay]
The University of Hong Kong

18. Reflective Media Diary (RMD)
• Identify tort-related events; track
developments; provide legal analysis;
• Portfolio-style: collecting, selecting &
editing material over time.
The University of Hong Kong

19. RMD features
Steady student engagement
Promotes reading habits
Incomplete, authentic problems
Workload friendly for teachers
The University of Hong Kong

20. Use of exemplars
Sharing of samples to illustrate expectations
The University of Hong Kong

21. Same-day exam feedback
Discussion immediately after exam
The University of Hong Kong

22. Exam debriefing
Supplemented by online discussion
Novel angles added to marking scheme
The University of Hong Kong

23. TASK DESIGN
PRINCIPLES
The University of Hong Kong

24. (1) Integrated with instruction and ILOs
The University of Hong Kong
Constructive
alignment

25. (2) Spreads student effort
The University of Hong Kong

26. (3) Mirrors real-life uses of the discipline
The University of Hong Kong

27. (4) Integrated and coherent
The University of Hong Kong

28. (5) Incorporates feedback dialogues
The University of Hong Kong

29. (6) Supports students in appreciating quality
The University of Hong Kong

30. (7) Flexibility & choice
The University of Hong Kong

31. (8) Encourages deep approaches to
learning
The University of Hong Kong

32. Summary of principles
1. Alignment
2. Spreading effort
3. Authenticity
4. Integrated/coherent
5. Feedback designs
6. Developing student connoisseurship
7. Flexibility or choice
8. Deep approaches to learning
The University of Hong Kong

33. Issues & Challenges
The University of Hong Kong

34. Double duty (Boud, 2000)
Competing priorities in
assessment
Reliability, Fairness,
Workload, QA etc
The University of Hong Kong

35. Marking & Grading
End of semester grading involves:
-Awarding a grade
-Justifying the grade
-Providing specific comments
-Providing generic comments
-Reciprocity
The University of Hong Kong

36. Staff frustrations
• Heavy marking load
• Students don’t collect feedback
• Students mainly interested in the grade
• Students lack motivation to act
…..
The University of Hong Kong

37. Student frustrations
Feedback often seems like a perversely belated
revelation of things that should have been made
clear earlier (Crook, Gross & Dymott, 2006)
The University of Hong Kong

38. Suggested shifts in priorities
The University of Hong Kong
Increase Decrease
In-class dialogic feedback within
module time
Unidirectional comments after
completion of module
Written feedback comments on
first assessment task of module
Written feedback comments on
final task of module
Feedback for first year students Feedback for final year students

39. Trust or distrust?
Distrust as barrier for assessment change
(Carless, 2009)
The University of Hong Kong

40. Teacher autonomy?
Many teachers lack autonomy: pulled in
different directions by competing priorities
(James, 2014)
The University of Hong Kong

41. CONCLUSION
The University of Hong Kong

42. Communication about assessment
Rationales
The University of Hong Kong
Potential benefits
Processes
Tackling challenges

43. Teacher X factor
Teacher determination to overcome barriers
& strive for learning-oriented assessment
The University of Hong Kong

44. References
Boud, D. (2000). Sustainable assessment: Rethinking assessment for the
learning society. Studies in Continuing Education, 22(2), 151-167.
Carless, D. (2009). Trust, distrust and their impact on assessment reform.
Assessment and Evaluation in Higher Education, 34(1), 79-89.
Carless, D. (2015). Exploring learning-oriented assessment processes. Higher
Education, 69(6), 963-976.
Carless, D. & J. Zhou (2015). Starting small in assessment change: Short in-
class written responses. Assessment and Evaluation in Higher Education,
41(7), 1114-1127
Crook, C., Gross, H. & Dymott, R. (2006). Assessment relationships in higher
education: The tension of process and practice. British Educational
Research Journal, 32(1), 95-114.
James, D. (2014). Investigating the curriculum through assessment practice in
higher education: The value of a ‘learning cultures’ approach. Higher
Education, 67(2), 155-169.
The University of Hong Kong

45. THANK YOU
The University of Hong Kong