Research in Distance Education: impact on practice conference, 27 October 2010. Presentation in Assessment Strand by Dr Harvey Mellar, Institute of Education.
More details at www.cde.london.ac.uk.
Model Call Girl in Tilak Nagar Delhi reach out to us at 🔝9953056974🔝
RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios
1. Formative e-assessment: case
stories, design patterns, and
future scenarios
Harvey Mellar
London Knowledge Lab
Institute of Education, University of London
http://feasst.wlecentre.ac.uk/
http://www.jisc.ac.uk/whatwedo/projects/feasst.aspx
2. Overview
Short term, scoping study commissioned by JISC, and supported by the Centre for
Excellence in Work-based Learning for Education Professionals
• Methodology
• Desk research
• Literature review
• Comparing frameworks
• Five Practical Enquiry Days
• Combination of collaborative reflection, report back from teams, and guest
plenaries
• Launch day, three Planet workshops, developers' day
(Adopted and adapted the Planet Project's Participatory Methodology for
Practical Design Patterns - http://patternlanguagenetwork.wordpress.com)
• Wiki for collaborative authoring of patterns
http://purl.org/planet/Groups.FormativeEAssessment/
4. A definition
“An assessment functions formatively when
evidence about student achievement elicited by
the assessment is interpreted and used to make
decisions about the next steps in instruction that
are likely to be better, or better founded, than
the decisions that would have been made in the
absence of that evidence”
(Dylan Wiliam)
5. Five strategies
Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment.
Educational Assessment, Evaluation and Accountability, 21(1), 5-31.
6. Moments of contingency
Teachers design their instruction to yield evidence about student
achievement, by carefully crafting hinge-point questions, for
example. These create ‘moments of
contingency’, in which the direction of
the instruction will depend on student
responses. Teachers provide feedback that engages
students, make time in class for students to work on
improvement, and activate students as instructional resources
for one another.
(Leahy, Lyon, Thompson and Wiliam, 2005)
9. 9
Pattern Mining Workshop
Shift from anecdotes
to transferable design
knowledge by
identifying
commonalities across
case stories, and
capturing them in a
semi-structured form
10. The core template
• Context
– Where, when, who (all the things you can’t change)
• Problem (pick one!)
– We want to do A under condition B but are
constrained by C
• Solution
(in any order that
works for you)
C o n t e x t
Problem Solution
When, Where,
Who
What are we trying to
achieve / solve?
Cookbook: ingredients,
procedure, expected
outcomes
14. Creature of the week
(Judy Robertson)
Situation
Large class (138), first and second year computer
science students. Assignment: create a virtual pet in
Second Life
Task
Engage and motivate the students
show examples of good work which others could
learn from
show students their work is valued.
build a sense of community.http://purl.org/planet/Cases/creatureoftheweek
15.
16. CoMo (Niall Winters, Yishay Mor)
Situation
Royal Veterinary College
Hospital rotations as part of the training
Task
Allow students to capture critical incidents in text
and image
Support sharing of clinical experiences and co-
reflection
http://purl.org/planet/Cases/CoMo
21. Problem
Lack of immediate feedback for students
leads to fossilisation of errors and
misconceptions
providing immediate feedback in an
iterative fashion can also hinder effective
learning since students are able to "grope
their way" step-by-step to a correct
solution without necessarily having to
think about each answer as a whole.
22. Context
Class size
Large (30-300)
Content
Skillsfacts
Mode of instruction
Blendedon-line. Computer tested
23. Solution
• Students are posed questions of a type which elicit answers that
can contain multiple errors
• If a student's answer is entirely correct a mark of 100% is
awarded
• If their answer contains errors, a mark is given which contributes
to a percentage of the total mark for the question, along with
detailed - yet generic - feedback on the location and type of the
errors
• Students are then permitted a second attempt in which to refine
their answer
• The mark for the 2nd attempt contributes to remaining
percentage of the total mark for the question
• Feedback on any remaining errors is also given, along with the
correct answer(s)
• No further attempts are permitted
25. Good feedback should
Alert learners to their weaknesses.
Diagnose the causes and dynamics of these.
Include operational suggestions to improve the learning
experience.
Address socio-emotive factors.
Tutors know this, but are pressed for
time, or are not aware of their feedback
strategies
Large teaching organisations are not equipped to
provide tutors with personal feedback on their
teaching
Problem
26. Context
Large scale, technology supported, graded
courses
many tutors instructing many students
Feedback is mediated by technology that allows it
to be captured and processed in real time
Topic of study is subject to both grading and
formative feedback
27. Solution
Embed a mechanism in the learning and teaching system
that regularly captures tutor feedback, analyses it, and
presents them with graphical representation of the types of
feedback they have given. Ideally, this should also include
constructive advice as to how to shift from less to more
effective forms.
In computer supported environments (e.g. VLEs), this
mechanism could be integrated into the system, providing
tutors with immediate analysis of their feedback, as well as
long-term aggregates.
29. High achievers
When using Try Once Refine Once, there is a risk
that high-achievers do not receive feedback
So
• Use Showcase Learning to celebrate students’
work and provoke feedback from peers and
tutors
• Use Feedback on Feedback to alert tutors to the
problem
31. Reminder of the five strategies
Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment.
Educational Assessment, Evaluation and Accountability, 21(1), 5-31.
35. Practical design patterns for
teaching and learning with
technology
A book for Sense Publisher's 'Technology
Enhanced Learning' series
Editors: Yishay Mor (London Knowledge Lab),
Steven Warburton (King's College London) and
Niall Winters (London Knowledge Lab)
http://www.practicalpatternsbook.org/Home
Notas do Editor
We define formative e-assessment as the use of ICT to support the iterative process of gathering and analysing information about student learning by teachers as well as learners and of evaluating it in relation to prior achievement and attainment of intended, as well as unintended learning outcomes, in a way that allows the teacher or student to adjust the learning trajectory
Key findings from the literature
The domain is complex and contentious:
there is a wide heterogeneity in the literature, and frequent slippage between terms such as ‘assessment’ and ‘learning’, ‘formative’ and ‘summative’ and there are widely differing theoretical emphases
a wide variety of perspectives and practices exist which prioritise different educational goals; components have been identified to reflect a variety of actors, learning intentions, roles and activities, and the mechanisms involved in enabling progression of learning towards measurable attributes
From: ‘practice’ assessment, or serial (or repeated) summative assessment
To: synonymous with learning
What does ‘e’ add?
Speed
Storage capacity
Processing
Communication
Construction and representation
Mutability
Adaptivity’ is a core component of e-assessment processes indicating the flexible responsiveness on the part of learners and teachers which may or may not itself involve the use of technology.
I. Speed
Speed of response is often important in enabling feedback to have an effect
Supports rapid iteration – in many cases the ability to give feedback quickly means that the student’s next problem solving iteration can begin more quickly.
II. Storage capacity
Ability to access very large amounts of data (so appropriate feedback/additional work/illustrations can be identified).
III. Processing
Automation – in some situations the e-assessment system can analyse responses automatically and provide appropriate feedback.
Scalability – can often be the result of some level of automation.
Adaptivity – systems can adapt to students.
V. Communication
Often the advantage of the ‘e’ is that it enables rapid communication of ideas across a range of audiences, and the technology allows this range to be controlled it can be just one person, a group, a class or more
This communication aspect means that aspects of communication can be captured and given a degree of semi-permanence
This semi permanence supports the sharing of intellectual objects.
V. Construction and representation
Representation – the ability to represent ideas in a variety of ways and to move and translate between these representations
Technology can support learners in the construction of representations of their own ideas.
By representation technology enables concepts to be ‘shaped’ and therefore affects their meaning, i.e. representation makes use of symbols which help meanings develop
In representing their ideas in digital artefacts (creating these intellectual objects) learners open up a window on their thinking.
VI. Mutability
Shared objects are not fixed, they can change/be changed easily and quickly.