Talk at Assessment Event, Durham University

1. Improving student learning
through programme assessment:
Interactive TESTA masterclass
@solentlearning
@tansyjtweets
Tansy Jessop
TESTA Project Leader
Durham University
3 May 2018

2. Masterclass Session 1
• Your assessment and feedback highs and lows
• Brief explanation of TESTA
• Why a programme approach?
• Balancing summative and formative
• Mock audit (plenary)
• DIT audit (Breakout rooms)

3. Your experience of powerful assessment
Think of a time when you learnt powerfully from
an assessment task. What made it powerful?
Go to www.menti.com and use the code 33 96 62
Enter three words or phrases which capture
something about it

4. Your main assessment and feedback
challenges
Go to www.menti.com and use the code 95 45 94
Type in three words or phrases which capture
your main challenges

5. Assessment and feedback are central to
student learning
1) Assessment drives what students pay attention
to, and defines the actual curriculum (Ramsden
2003).
2) Feedback is the single most important factor in
learning (Hattie 2009; Black and Wiliam 1998).

7. Research and change process
Programme
Team
Meeting
Assessment
Experience
Questionnaire
(AEQ)
TESTA
Programme
Audit
Student
Focus Groups

8. Sustained growth

9. TESTA….
“…is a way of thinking
about assessment and
feedback”
Graham Gibbs

10. Why take a programme approach?
1. A modular problem
2. A curriculum problem
3. An alienation problem
4. A context problem
5. A counter-narrative

11. The modular degree

12. IKEA 101: great for flat-pack furniture but..

13. Curriculum privileges knowing stuff

14. The best approach from the student’s perspective is to focus
on concepts. I’m sorry to break it to you, but your students are
not going to remember 90 per cent – possibly 99 per cent – of
what you teach them unless it’s conceptual…. when broad,
over-arching connections are made, education occurs. Most
details are only a necessary means to that end.
http://www.timeshighereducation.co.uk/features/a-students-
lecture-to-rofessors/2013238.fullarticle#.U3orx_f9xWc.twitter
A student’s lecture to her professor

15. A state of alienation?
Image, "Alienation Nightmare" © 1996 by Sabu

16. Motorways to alienation
• M1: Modules
• M2: Markets
• M3: Metrics
• M4: Mass higher education

17. TESTA improves students’ perceptions
of A&F…
60%
65%
70%
75%
80%
85%
90%
95%
Q5 Q6 Q7 Q8 Q9 OS
AVERAGENSSSCORES
COMPARISON OF 32 PROGS IN 13 UNIVERSITIES WITH SECTOR SCORES
NSS 2015 SCORES TESTA SCORES

18. …and improves the staff experience
• Reduce summative assessments
(eg. From 48-24)
• More challenging and interesting
summative tasks
• More engaging sessions with
students participating in formative
• Better connections across the
curriculum
• Less content-driven curriculum,
and more process-oriented

19. Activity One: mock audit
Programme
Team
Meeting
Assessment
Experience
Questionnaire
(AEQ)
TESTA
Programme
Audit
Student
Focus Groups

20. The Audit: Caveats
1. Audit is not everything
2. Official discourse
3. Planned curriculum
4. Some better data, some weaker, some gaps

21. The Audit data
• Number of assessment tasks
• Summative/formative
• Variety
• Proportion of exams
• Written feedback
• Speed of return of feedback
• Some context

22. TESTA definitions
Summative:
graded assessment which counts towards the degree
Formative:
Does not count: ungraded, required task with
feedback

23. Mock Audit

24. DIY audit
• Form groups of 8, preferably with some
people from same discipline
• Flipchart and pens, ten step guide, and go to
a break out room
• Choose a programme and a facilitator from
outside that programme
• Run an audit on one level of the programme
• You have 20 minutes
• Come back here!

25. Audit wrap up
• How did it go?
• Any problems?
• Any questions?
• Any insights?
• Any refinements?

26. Assessment features across a 3 year UG degree (n=73)
Characteristic Range
Summative 12 -227
Formative 0 - 116
Varieties of assessment 5 - 21
Proportion of examinations 0% - 87%
Time to return marks & feedback 10 - 42 days
Volume of oral feedback 37 -1800 minutes
Volume of written feedback 936 - 22,000 words

27. Typical A&F patterns
Characteristic Low Medium High
Volume of summative
assessment
Below 33 40-48 More than 48
Volume of formative only Below 1 5-19 More than 19
% of tasks by examinations Below 11% 22-31% More than 31%
Variety of assessment
methods
Below 8 11-15 More than 15
Written feedback in words Less than 3,800 6,000-7,600 More than 7,600

28. Actions based on evidence
a) Reduction in summative
b) Increase in formative
c) Streamlined varieties
d) More or less feedback depending…
e) Quantifiable
f) Every time a coconut with each feature

29. Theme 1: High summative: low formative
• Summative ‘pedagogies of control’
• Circa 2 per module in UK
• Ratio of 1:8 of formative to summative
• Formative weakly understood and practised

30. Assessment Arms Race

31. A lot of people don’t do wider
reading. You just focus on your
essay question.
In Weeks 9 to 12 there is hardly
anyone in our lectures. I'd rather
use those two hours of lectures
to get the assignment done.
It’s been non-stop
assignments, and I’m now
free of assignments until
the exams – I’ve had to
rush every piece of work
I’ve done.
CONSEQUENCES
OF HIGH
SUMMATIVE

32. It was really useful. We were assessed
on it but we weren’t officially given a
grade, but they did give us feedback on
how we did.
It didn’t actually count so that
helped quite a lot because it
was just a practice and didn’t
really matter what we did and
we could learn from mistakes so
that was quite useful.
WHAT ABOUT FORMATIVE?

33. If there weren’t loads
of other assessments,
I’d do it.
It’s good to know you’re
being graded because
you take it more
seriously.
BUT… If there are no actual
consequences of not doing
it, most students are going
to sit in the bar.
The lecturers do formative
assessment but we don’t get
any feedback on it.

34. Why it is broken
1. HE factory mass produced goods
2. Summative ‘pedagogy of control’
3. Ideas about student autonomy
get in the way
4. Not much conviction about
formative – lip service
5. Not programme-wide
“Between the idea
And the reality
Between the motion
And the act
Falls the Shadow”
T S Eliot, The Hollow Men

35. 1) Low-risk way of learning from feedback (Sadler, 1989)
2) Fine-tune understanding of goals (Boud 2000, Nicol 2006)
3) Feedback to lecturers to adapt teaching (Hattie, 2009)
4) Cycles of reflection and collaboration (Biggs 2003; Nicol &
McFarlane Dick 2006)
5) Encourages and distributes student effort (Gibbs 2004).
Why formative matters

36. How to encourage formative
Go to www.menti.com and use the code 79 27 56
Choose your top three strategies for engaging
students in formative assessment

37. Case Study 1
• Systematic reduction of summative across
whole business school
• Systematic ramping up of formative
• All working to similar script
• Systematic shift, experimentation, less risky
together

38. Case Study 2
• Formative presentations
• Students get feedback (peer and tutor)
• Refines their thinking for…
• Conceptually linked summative essay

39. Case Study 3
• Problem: silent seminar, students not reading
• Public platform blogging
• Current academic texts
• In-class
• Threads and live discussion
• Linked to summative

40. Case Study 4
• Problem: lack of discrimination about sources
• Students bring 1 x book, 1 x chapter, 1 x
journal article, 2 x pop culture articles to
seminar
• Justify choices to group
• Reach consensus about five best sources
• Add to reading list

41. Case Study 5
https://www.youtube.com/watch?v=ZVFwQzlVFy0

42. Your principles
(from memory so with gaps)
• Collaborative, involves people and team work
• Public
• Makes links between formative and
summative but different from a mock
• Authentic research tasks
• Iterative – not one off

43. Your task
• In groups, identify five principles for making
formative work. Write them down on flipchart
paper.
• Devise one or two adaptations for your
discipline, using the principles, your experience
and the case studies.
• Add one adaptation to your poster that you
might use on your programme.

44. References
Barlow, A. and Jessop, T. 2016. “You can’t write a load of rubbish”: Why blogging works as formative
assessment. Educational Developments. 17(3), 12-15. SEDA.
Boud, D. and Molloy, E. (2013) ‘Rethinking models of feedback for learning: The challenge of
design’, Assessment & Evaluation in Higher Education, 38(6), pp. 698–712.
Gibbs, G. & Simpson, C. (2004) Conditions under which assessment supports students' learning. Learning
and Teaching in Higher Education. 1(1): 3-31.
Harland, T., McLean, A., Wass, R., Miller, E. and Sim, K. N. (2014) ‘An assessment arms race and its fallout:
High-stakes grading and the case for slow scholarship’, Assessment & Evaluation in Higher Education.
Jessop, T. and Tomas, C. 2017. The implications of programme assessment on student learning. Assessment
and Evaluation in Higher Education.
Jessop, T. and Maleckar, B. 2016. The Influence of disciplinary assessment patterns on student learning: a
comparative study. Studies in Higher Education. Published Online 27 August 2014
Jessop, T. , El Hakim, Y. and Gibbs, G. (2014) The whole is greater than the sum of its parts: a large-scale
study of students’ learning in response to different assessment patterns. Assessment and Evaluation in
Higher Education. 39(1) 73-88.
Nicol, D. 2010. From monologue to dialogue: improving written feedback processes in mass higher
education, Assessment & Evaluation in Higher Education, 35: 5, 501 – 517.
O'Donovan, B , Price, M. and Rust, C. (2008) 'Developing student understanding of assessment standards: a
nested hierarchy of approaches', Teaching in Higher Education, 13: 2, 205 -217.
Sadler, D. R. 1989. ‘Formative assessment and the design of instructional systems’, Instructional Science,
18(2), pp. 119–144.
Tomas, C and Jessop, T. 2018. Struggling and juggling: A comparison of student assessment loads across
research and teaching-intensive universities. Assessment and Evaluation in Higher Education. 18 April.
Wu, Q. and Jessop, T. 2018. Formative assessment: missing in action in both research-intensive and teaching-
focused universities. Assessment and Evaluation in Higher Education. Published online 15 January.