Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Out of the long shadow of the NSS: TESTA's transformative potential
1. Out of the long shadow of the NSS:
TESTA’s transformative potential
@solentlearning
@tansyjtweets
Tansy Jessop
TESTA Project Leader
16 November 2017
9. Three problems TESTA addresses
1. Things going awry but not sure why
2. Curriculum design challenge
3. Academic reading and writing challenge
10. It was heavy, tons of
marking for the tutor. It
was such hard work. It was
criminal.
Media Course Leader
I’m really bad at reading
feedback. I’ll look at the mark
and then be like ‘well stuff it, I
can’t do anything about it’
Student, TESTA focus group
1. Things going awry, not sure why
17. The best approach from the student’s perspective is to focus
on concepts. I’m sorry to break it to you, but your students are
not going to remember 90 per cent – possibly 99 per cent – of
what you teach them unless it’s conceptual…. when broad,
over-arching connections are made, education occurs. Most
details are only a necessary means to that end.
http://www.timeshighereducation.co.uk/features/a-students-
lecture-to-rofessors/2013238.fullarticle#.U3orx_f9xWc.twitter
A student’s lecture to her professor
19. Is assessment environment prompting
it enough?
Significant learning
gains for students who
1) Read > 40 pages a
week of academic
writing
2) Write > 20 pages per
semester for each unit
21. TESTA evidence and strategies for…
1. Variations in assessment patterns
2. High summative and low formative diets
3. Disconnected feedback
4. Confusion about goals and standards
23. 1. Huge variations
• What is striking for
you about this data?
• How does it compare
with your context?
• Does variation
matter?
24. Assessment features across a 3 year UG degree (n=73)
Characteristic Range
Summative 12 -227
Formative 0 - 116
Varieties of assessment 5 - 21
Proportion of examinations 0% - 87%
Time to return marks & feedback 10 - 42 days
Volume of oral feedback 37 -1800 minutes
Volume of written feedback 936 - 22,000 words
25. Typical A&F patterns
Characteristic Low Medium High
Volume of summative
assessment
Below 33 40-48 More than 48
Volume of formative only Below 1 5-19 More than 19
% of tasks by examinations Below 11% 22-31% More than 31%
Variety of assessment
methods
Below 8 11-15 More than 15
Written feedback in words Less than 3,800 6,000-7,600 More than 7,600
26. Actions based on evidence
a) Reduction in summative
b) Increase in formative
c) Streamlined varieties
d) More or less feedback depending…
e) Quantifiable
f) Every time a coconut with each feature
27. Theme 2: High summative: low formative
• Summative ‘pedagogies of control’
• Circa 2 per module in UK
• Ratio of 1:8 of formative to summative
• Formative weakly understood and practised
29. A lot of people don’t do wider
reading. You just focus on your
essay question.
In Weeks 9 to 12 there is hardly
anyone in our lectures. I'd rather
use those two hours of lectures
to get the assignment done.
It’s been non-stop
assignments, and I’m now
free of assignments until
the exams – I’ve had to
rush every piece of work
I’ve done.
CONSEQUENCES
OF HIGH
SUMMATIVE
30. It was really useful. We were assessed
on it but we weren’t officially given a
grade, but they did give us feedback on
how we did.
It didn’t actually count so that
helped quite a lot because it
was just a practice and didn’t
really matter what we did and
we could learn from mistakes so
that was quite useful.
WHAT ABOUT FORMATIVE?
31. If there weren’t loads
of other assessments,
I’d do it.
It’s good to know you’re
being graded because
you take it more
seriously.
BUT… If there are no actual
consequences of not doing
it, most students are going
to sit in the bar.
The lecturers do formative
assessment but we don’t get
any feedback on it.
32. So, how do engage students in
meaningful formative?
Go to www.menti.com and use the code 24 59 70
Choose the strategies you think work best
33. Case Study 1
• Systematic reduction of summative across
whole business school
• Systematic ramping up of formative
• All working to similar script
• Systematic shift, experimentation, less risky
together
34. Case Study 2
• Problem: silent seminar, students not reading
• Public platform blogging
• Current academic texts
• In-class
• Threads and live discussion
• Linked to summative
38. System-wide problems
• Modular structures impede feedback
• Students don’t feel known
• There is very little formative feedback
• Lecturers do too much judging; students too little
39. Mass HE is squeezing out dialogue
with the result that written feedback,
which is essentially a one-way communication,
has to carry almost all the burden of
student-teacher interaction
David Nicol 2010
41. Irretrievable breakdown…
Your essay lacked structure and
your referencing is problematic
Your classes are boring and I
don’t really like you
42. An impoverished dialogue
The many diverse
expressions of
dissatisfaction with
feedback can be taken as
symptoms of an
impoverished and
fractured dialogue
David Nicol 2010
43. Take five
• Choose a quote that
strikes you.
• What is the key issue?
• What strategies might
address this issue?
44. The feedback is
generally focused
on the module
Because it’s at the end
of the module, it doesn’t
feed into our future
work.
If It’s difficult because your
assignments are so detached
from the next one you do for
that subject. They don’t
relate to each other.
I read it and think “Well,
that’s fine but I’ve already
handed it in now and got the
mark. It’s too late”.
STRUCTURAL
45. It was like ‘Who’s
Holly?’ It’s that
relationship where
you’re just a student.
Because they have to mark so
many that our essay becomes
lost in the sea that they have
to mark.
Here they say ‘Oh yes, I don’t
know who you are. Got too
many to remember, don’t
really care, I’ll mark you on
your assignment’.
RELATIONAL
46. Ways to be dialogic
• Conversation: who starts the dialogue?
• Cycles of reflection across modules
• Quick generic feedback
• Feedback synthesis tasks
• Peer feedback (especially on formative)
• Technology: audio, screencast and blogging
• From feedback as ‘telling’…
• … to feedback as asking questions
48. Students to lecturers:
Critical Incident Questionnaire
Stephen Brookfield’s Critical Incident Questionnaire http://bit.ly/1loUzq0
49. Theme 4: Confusion about goals and
standards
• Consistently low scores on the AEQ for clear
goals and standards
• Alienation from the tools, especially criteria
and guidelines
• Symptoms: perceptions of marker variation,
unfair standards and inconsistencies in practice
50. We’ve got two
tutors- one marks
completely differently
to the other and it’s
pot luck which one
you get.
They read the essay and then
they get a general impression,
then they pluck a mark from
the air.
It’s like Russian
roulette – you may
shoot yourself and
then get an A1.
They have different
criteria, they build up their
own criteria.
51. There are criteria, but I find them really
strange. There’s “writing coherently,
making sure the argument that you
present is backed up with evidence”.
55. The Art and Science of Evaluation
Judging is both an art and a science: It is an art because
the decisions with which a judge is constantly faced are
very often based on considerations of an intangible
nature that cannot be recognized intuitively. It is also a
science because without a sound knowledge of a dog’s
points and anatomy, a judge cannot make a proper
assessment of it whether it is standing or in motion.
Take them round please: the art of judging dogs (Horner, 1975).
56. Taking action: internalising goals and
standards
• Regular calibration exercises
• Discussion and dialogue
• Discipline specific criteria (no cut and paste)
Lecturers
• Rewrite/co-create criteria
• Marking exercises
• Discussing exemplars
Lecturers
and students
• Enter secret garden - peer review
• Engage in drafting processes
• Self-reflection
Students
57.
58. References
Arum R. and Roksa J. 2011. Academically Adrift: Limited Learning on College Campuses. University of Chicago
Press.
Barlow, A. and Jessop, T. 2016. “You can’t write a load of rubbish”: Why blogging works as formative
assessment. Educational Development. 17(3), 12-15. SEDA.
Boud, D. and Molloy, E. (2013) ‘Rethinking models of feedback for learning: The challenge of
design’, Assessment & Evaluation in Higher Education, 38(6), pp. 698–712.
Carr,. N. 2010. The Shallows. London. Atlantic Books
Gibbs, G. & Simpson, C. (2004) Conditions under which assessment supports students' learning. Learning
and Teaching in Higher Education. 1(1): 3-31.
Harland, T., McLean, A., Wass, R., Miller, E. and Sim, K. N. (2014) ‘An assessment arms race and its fallout:
High-stakes grading and the case for slow scholarship’, Assessment & Evaluation in Higher Education.
Jessop, T. and Tomas, C. 2016 The implications of programme assessment on student learning. Assessment
and Evaluation in Higher Education. Published online 2 August 2016.
Jessop, T. and Maleckar, B. (2014). The Influence of disciplinary assessment patterns on student learning: a
comparative study. Studies in Higher Education. Published Online 27 August 2014
http://www.tandfonline.com/doi/abs/10.1080/03075079.2014.943170
Jessop, T. , El Hakim, Y. and Gibbs, G. (2014) The whole is greater than the sum of its parts: a large-scale
study of students’ learning in response to different assessment patterns. Assessment and Evaluation in
Higher Education. 39(1) 73-88.
Nicol, D. (2010) From monologue to dialogue: improving written feedback processes in mass higher
education, Assessment & Evaluation in Higher Education, 35: 5, 501 – 517.
O'Donovan, B , Price, M. and Rust, C. (2008) 'Developing student understanding of assessment standards: a
nested hierarchy of approaches', Teaching in Higher Education, 13: 2, 205 -217.
Sadler, D. R. (1989) ‘Formative assessment and the design of instructional systems’, Instructional Science,
18(2), pp. 119–144. doi: 10.1007/bf00117714.
Notas do Editor
Tansy
How do you measure soft stuff? 5 day cricket match versus 20/20
What started as a research methodology has become a way of thinking. David Nicol – changing the discourse, the way we think about assessment and feedback; not only technical, research, mapping, also shaping our thinking. Evidence, assessment principles. Habermas framework.
Academics operate in isolation from one another. Only see their part of the degree. Don’t see connections. Fragments into small tasks – hamster wheel. Curriculum design issue. The trouble is that students experience the whole elephant and it is often indigestible… Assessment is mainly sort of the topical knowledge and the topics never relate. We'll never do something again that we’ve already studied, like we learn something and then just move on
(TESTA focus group data).
Feedback: all that effort, but what is the effect? Margaret Price. Feedback more important to teachers than students…
But lots of projects and programmes do….
Hard to make connections, difficult to see the joins between assessments, much more assessment, much more assessment to accredit each little box. Multiplier effect. Less challenge, less integration. Lots of little neo-liberal tasks. The Assessment Arms Race.
Language of ‘covering material’ Should we be surprised?
Exeter economics/
Teach Less, learn more. Assess less, learn more.
Is anyone listening?
Students can increase their understanding of the language of assessment through their active engagement in: ‘observation, imitation, dialogue and practice’ (Rust, Price, and O’Donovan 2003, 152), Dialogue, clever strategies, social practice, relationship building, relinquishing power.